Trace number 3741124

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB11: borg pb-dec-11.04.03? (MO) 307.86 307.963

General information on the benchmark

Name/DEC-SMALLINT-LIN/lopes/
normalized-31.opb
MD5SUM1231d9e508dc821e38b22e6db3fd2b2f
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark0
Best CPU time to get the best result obtained on this benchmark203.957
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables9310
Total number of constraints199239
Number of constraints which are clauses178407
Number of constraints which are cardinality constraints (but not clauses)730
Number of constraints which are nor clauses,nor cardinality constraints20102
Minimum length of a constraint1
Maximum length of a constraint231
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 3
Number of bits of the biggest number in a constraint 2
Biggest sum of numbers in a constraint 232
Number of bits of the biggest sum of numbers8
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.39/1.25	c INFO: loaded portfolio from HOME/etc/borg-mix+class.dec.pickle
0.39/1.25	c INFO: solving HOME/instance-3741124-1338721247.opb
1.78/2.69	c INFO: parsing took 1.39 s
4.18/5.03	c INFO: features took 2.34 CPU seconds
4.18/5.06	c INFO: running wbo1.4b-fixed@0 for 45 with 1795 remaining (b = 0.96)
54.62/55.40	c INFO: running pbct-0.1.2-linear@0 for 45 with 1746 remaining (b = 0.96)
102.28/103.09	c INFO: running wbo1.4a@0 for 45 with 1699 remaining (b = 0.94)
163.94/164.63	c INFO: wrote minion input file to HOME/tmpThiGRa.minion
163.94/164.63	c INFO: running minion-0.12@0 for 45 with 1650 remaining (b = 0.71)
211.59/212.30	c INFO: running bsolo_pb10-l2@0 for 45 with 1592 remaining (b = 0.65)
259.63/260.21	c INFO: running clasp-1.3.7@0 for 45 with 1545 remaining (b = 0.62)
306.73/307.38	c INFO: running sat4j-pb-v20101225-cutting@0 for 45 with 1498 remaining (b = 0.22)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3741124-1338721247/watcher-3741124-1338721247 -o /tmp/evaluation-result-3741124-1338721247/solver-3741124-1338721247 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3741124-1338721247.opb -seed 1895152942 -budget 1800 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.00 4.05 3.21 1/181 6540
/proc/meminfo: memFree=18072568/32873844 swapFree=7116/7116
[pid=6540] ppid=6538 vsize=15276 CPUtime=0 cores=1,3,5,7
/proc/6540/stat : 6540 (python) D 6538 6540 2614 0 -1 4202496 634 0 0 0 0 0 0 0 20 0 1 0 338250222 15642624 527 33554432000 4194304 5683644 140733316784832 140733316770744 240565175184 0 0 16781312 2 18446744072100609316 0 0 17 1 0 0 0 0 0
/proc/6540/statm: 3819 527 312 364 0 261 0

[startup+0.027697 s]
/proc/loadavg: 5.00 4.05 3.21 1/181 6540
/proc/meminfo: memFree=18072568/32873844 swapFree=7116/7116
[pid=6540] ppid=6538 vsize=15948 CPUtime=0 cores=1,3,5,7
/proc/6540/stat : 6540 (python) D 6538 6540 2614 0 -1 4202496 882 0 0 0 0 0 0 0 20 0 1 0 338250222 16330752 738 33554432000 4194304 5683644 140733316784832 140733316711592 240565175184 0 0 16781312 2 18446744072100609316 0 0 17 1 0 0 0 0 0
/proc/6540/statm: 3987 738 347 364 0 429 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15948

[startup+0.100309 s]
/proc/loadavg: 5.00 4.05 3.21 1/181 6540
/proc/meminfo: memFree=18072568/32873844 swapFree=7116/7116
[pid=6540] ppid=6538 vsize=19084 CPUtime=0.03 cores=1,3,5,7
/proc/6540/stat : 6540 (python) R 6538 6540 2614 0 -1 4202496 1395 0 0 0 2 1 0 0 20 0 1 0 338250222 19542016 1149 33554432000 4194304 5683644 140733316784832 140733316715336 4602989 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/6540/statm: 4771 1149 430 364 0 695 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 19084

[startup+0.300278 s]
/proc/loadavg: 5.00 4.05 3.21 1/181 6540
/proc/meminfo: memFree=18072568/32873844 swapFree=7116/7116
[pid=6540] ppid=6538 vsize=96896 CPUtime=0.11 cores=1,3,5,7
/proc/6540/stat : 6540 (python) D 6538 6540 2614 0 -1 4202496 3950 1012 1 0 8 3 0 0 20 0 1 0 338250222 99221504 2649 33554432000 4194304 5683644 140733316784832 140733316666920 240565175184 0 0 16781312 2 18446744072100609316 0 0 17 1 0 0 0 0 0
/proc/6540/statm: 24224 2649 823 364 0 1715 0
Current children cumulated CPU time (s) 0.11
Current children cumulated vsize (KiB) 96896

[startup+0.700211 s]
/proc/loadavg: 5.00 4.05 3.21 1/181 6540
/proc/meminfo: memFree=18072568/32873844 swapFree=7116/7116
[pid=6540] ppid=6538 vsize=134136 CPUtime=0.23 cores=1,3,5,7
/proc/6540/stat : 6540 (python) D 6538 6540 2614 0 -1 4202496 6213 1012 1 0 15 8 0 0 20 0 1 0 338250222 137355264 4053 33554432000 4194304 5683644 140733316784832 140733316661608 240565175184 0 0 16781312 2 18446744072100609316 0 0 17 1 0 0 0 0 0
/proc/6540/statm: 33534 4053 1084 364 0 2952 0
Current children cumulated CPU time (s) 0.23
Current children cumulated vsize (KiB) 134136

[startup+1.50025 s]
/proc/loadavg: 5.00 4.05 3.21 2/182 6545
/proc/meminfo: memFree=18055960/32873844 swapFree=7116/7116
[pid=6540] ppid=6538 vsize=296980 CPUtime=0.67 cores=1,3,5,7
/proc/6540/stat : 6540 (python) R 6538 6540 2614 0 -1 4202496 12365 1012 1 0 53 14 0 0 20 0 1 0 338250222 304107520 10931 33554432000 4194304 5683644 140733316784832 140733316778072 240564801428 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/6540/statm: 74245 10931 1811 364 0 9474 0
Current children cumulated CPU time (s) 0.67
Current children cumulated vsize (KiB) 296980

[startup+3.10025 s]
/proc/loadavg: 5.00 4.05 3.21 2/182 6545
/proc/meminfo: memFree=18015352/32873844 swapFree=7116/7116
[pid=6540] ppid=6538 vsize=335156 CPUtime=2.28 cores=1,3,5,7
/proc/6540/stat : 6540 (python) R 6538 6540 2614 0 -1 4202496 26551 1012 1 0 210 18 0 0 20 0 1 0 338250222 343199744 20541 33554432000 4194304 5683644 140733316784832 140733316777448 139730483638539 0 0 16781312 33554434 0 0 0 17 1 0 0 0 0 0
/proc/6540/statm: 83789 20541 1846 364 0 19018 0
Current children cumulated CPU time (s) 2.28
Current children cumulated vsize (KiB) 335156
heavy processes:

[startup+6.30043 s]
/proc/loadavg: 5.00 4.06 3.21 2/185 6548
/proc/meminfo: memFree=17981912/32873844 swapFree=7116/7116
[pid=6540] ppid=6538 vsize=415628 CPUtime=4.24 cores=1,3,5,7
/proc/6540/stat : 6540 (python) S 6538 6540 2614 0 -1 4202496 28005 1012 1 0 406 18 0 0 20 0 2 0 338250222 425603072 21770 33554432000 4194304 5683644 140733316784832 140733316779664 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/6540/statm: 103907 21770 1902 364 0 39130 0
[pid=6546] ppid=6540 vsize=339848 CPUtime=0 cores=1,3,5,7
/proc/6546/stat : 6546 (python) S 6540 6540 2614 0 -1 4202560 744 0 0 0 0 0 0 0 20 0 1 0 338250729 348004352 20202 33554432000 4194304 5683644 140733316784832 140733316775920 240565191080 0 0 16781312 33554946 18446744071580425449 0 0 17 5 0 0 0 0 0
/proc/6546/statm: 84962 20202 340 364 0 20185 0
[pid=6548] ppid=6546 vsize=23876 CPUtime=1.22 cores=1,3,5,7
/proc/6548/stat : 6548 (wbo1.4b-fixed) R 6546 6548 6548 0 -1 4202496 7919 0 0 0 117 5 0 0 20 0 1 0 338250729 24449024 5386 33554432000 134512640 135752315 4291332368 4291330444 135270849 0 0 16781312 3 0 0 0 17 1 0 0 0 0 0
/proc/6548/statm: 5969 5386 146 303 0 5664 0
[pid=6540/tid=6547] ppid=6538 vsize=415628 CPUtime=0 cores=1,3,5,7
/proc/6540/task/6547/stat : 6547 (python) S 6538 6540 2614 0 -1 4202560 26 1012 0 0 0 0 0 0 20 0 2 0 338250729 425603072 21770 33554432000 4194304 5683644 140733316784832 139730479198768 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 5.46
Current children cumulated vsize (KiB) 779352

[startup+12.7005 s]
/proc/loadavg: 5.00 4.08 3.22 2/185 6571
/proc/meminfo: memFree=17964548/32873844 swapFree=7116/7116
[pid=6540] ppid=6538 vsize=415628 CPUtime=4.24 cores=1,3,5,7
/proc/6540/stat : 6540 (python) S 6538 6540 2614 0 -1 4202496 28005 1012 1 0 406 18 0 0 20 0 2 0 338250222 425603072 21770 33554432000 4194304 5683644 140733316784832 140733316779664 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/6540/statm: 103907 21770 1902 364 0 39130 0
[pid=6546] ppid=6540 vsize=339848 CPUtime=0 cores=1,3,5,7
/proc/6546/stat : 6546 (python) S 6540 6540 2614 0 -1 4202560 883 0 0 0 0 0 0 0 20 0 1 0 338250729 348004352 20225 33554432000 4194304 5683644 140733316784832 140733316775920 240565191080 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/6546/statm: 84962 20225 363 364 0 20185 0
[pid=6548] ppid=6546 vsize=29044 CPUtime=7.62 cores=1,3,5,7
/proc/6548/stat : 6548 (wbo1.4b-fixed) R 6546 6548 6548 0 -1 4202496 9212 0 0 0 734 28 0 0 20 0 1 0 338250729 29741056 6679 33554432000 134512640 135752315 4291332368 4291330444 135270849 0 0 16781312 3 0 0 0 17 1 0 0 0 0 0
/proc/6548/statm: 7261 6679 146 303 0 6956 0
[pid=6540/tid=6547] ppid=6538 vsize=415628 CPUtime=0 cores=1,3,5,7
/proc/6540/task/6547/stat : 6547 (python) S 6538 6540 2614 0 -1 4202560 26 1012 0 0 0 0 0 0 20 0 2 0 338250729 425603072 21770 33554432000 4194304 5683644 140733316784832 139730479198768 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 11.86
Current children cumulated vsize (KiB) 784520

[startup+25.5004 s]
/proc/loadavg: 5.00 4.12 3.25 2/185 6571
/proc/meminfo: memFree=17955096/32873844 swapFree=7116/7116
[pid=6540] ppid=6538 vsize=415628 CPUtime=4.24 cores=1,3,5,7
/proc/6540/stat : 6540 (python) S 6538 6540 2614 0 -1 4202496 28005 1012 1 0 406 18 0 0 20 0 2 0 338250222 425603072 21770 33554432000 4194304 5683644 140733316784832 140733316779664 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/6540/statm: 103907 21770 1902 364 0 39130 0
[pid=6546] ppid=6540 vsize=339848 CPUtime=0.02 cores=1,3,5,7
/proc/6546/stat : 6546 (python) S 6540 6540 2614 0 -1 4202560 894 0 0 0 1 1 0 0 20 0 1 0 338250729 348004352 20225 33554432000 4194304 5683644 140733316784832 140733316775920 240565191080 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/6546/statm: 84962 20225 363 364 0 20185 0
[pid=6548] ppid=6546 vsize=39500 CPUtime=20.41 cores=1,3,5,7
/proc/6548/stat : 6548 (wbo1.4b-fixed) R 6546 6548 6548 0 -1 4202496 11816 0 0 0 1991 50 0 0 20 0 1 0 338250729 40448000 9283 33554432000 134512640 135752315 4291332368 4291330444 135270849 0 0 16781312 3 0 0 0 17 1 0 0 0 0 0
/proc/6548/statm: 9875 9283 146 303 0 9570 0
[pid=6540/tid=6547] ppid=6538 vsize=415628 CPUtime=0 cores=1,3,5,7
/proc/6540/task/6547/stat : 6547 (python) S 6538 6540 2614 0 -1 4202560 26 1012 0 0 0 0 0 0 20 0 2 0 338250729 425603072 21770 33554432000 4194304 5683644 140733316784832 139730479198768 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 24.67
Current children cumulated vsize (KiB) 794976

[startup+51.1004 s]
/proc/loadavg: 4.36 4.05 3.25 3/185 6576
/proc/meminfo: memFree=24053404/32873844 swapFree=7116/7116
[pid=6540] ppid=6538 vsize=415628 CPUtime=4.24 cores=1,3,5,7
/proc/6540/stat : 6540 (python) S 6538 6540 2614 0 -1 4202496 28005 1012 1 0 406 18 0 0 20 0 2 0 338250222 425603072 21770 33554432000 4194304 5683644 140733316784832 140733316779664 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/6540/statm: 103907 21770 1902 364 0 39130 0
[pid=6546] ppid=6540 vsize=339848 CPUtime=0.06 cores=1,3,5,7
/proc/6546/stat : 6546 (python) S 6540 6540 2614 0 -1 4202560 995 0 0 0 3 3 0 0 20 0 1 0 338250729 348004352 20230 33554432000 4194304 5683644 140733316784832 140733316775920 240565191080 0 0 16781312 33554946 18446744071580425449 0 0 17 1 0 0 0 0 0
/proc/6546/statm: 84962 20230 368 364 0 20185 0
[pid=6548] ppid=6546 vsize=60244 CPUtime=46 cores=1,3,5,7
/proc/6548/stat : 6548 (wbo1.4b-fixed) R 6546 6548 6548 0 -1 4202496 16982 0 0 0 4528 72 0 0 20 0 1 0 338250729 61689856 14449 33554432000 134512640 135752315 4291332368 4291330444 135270849 0 0 16781312 3 0 0 0 17 5 0 0 0 0 0
/proc/6548/statm: 15061 14449 146 303 0 14756 0
[pid=6540/tid=6547] ppid=6538 vsize=415628 CPUtime=0 cores=1,3,5,7
/proc/6540/task/6547/stat : 6547 (python) S 6538 6540 2614 0 -1 4202560 26 1012 0 0 0 0 0 0 20 0 2 0 338250729 425603072 21770 33554432000 4194304 5683644 140733316784832 139730479198768 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 50.3
Current children cumulated vsize (KiB) 815720

[startup+102.307 s]
/proc/loadavg: 3.02 3.73 3.18 3/192 6584
/proc/meminfo: memFree=23212276/32873844 swapFree=7116/7116
[pid=6540] ppid=6538 vsize=491420 CPUtime=4.27 cores=1,3,5,7
/proc/6540/stat : 6540 (python) S 6538 6540 2614 0 -1 4202496 28938 1012 1 0 408 19 0 0 20 0 3 0 338250222 503214080 21779 33554432000 4194304 5683644 140733316784832 140733316779664 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 7 0 0 0 0 0
/proc/6540/statm: 122855 21779 1905 364 0 58075 0
[pid=6546] ppid=6540 vsize=415628 CPUtime=0.08 cores=1,3,5,7
/proc/6546/stat : 6546 (python) S 6540 6540 2614 0 -1 4202560 1131 0 0 0 4 4 0 0 20 0 2 0 338250729 425603072 20265 33554432000 4194304 5683644 140733316784832 140733316776480 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/6546/statm: 103907 20265 396 364 0 39130 0
[pid=6548] ppid=6546 vsize=62580 CPUtime=50.29 cores=1,3,5,7
/proc/6548/stat : 6548 (wbo1.4b-fixed) T 6546 6548 6548 0 -1 4202496 17485 0 0 0 4954 75 0 0 20 0 1 0 338250729 64081920 14952 33554432000 134512640 135752315 4291332368 4291330368 134611404 0 0 16781312 3 18446744071579387992 0 0 17 5 0 0 0 0 0
/proc/6548/statm: 15645 14952 146 303 0 15340 0
[pid=6546/tid=6577] ppid=6540 vsize=415628 CPUtime=0 cores=1,3,5,7
/proc/6546/task/6577/stat : 6577 (python) S 6540 6540 2614 0 -1 4202560 43 0 0 0 0 0 0 0 20 0 2 0 338255761 425603072 20265 33554432000 4194304 5683644 140733316784832 139730479198768 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6578] ppid=6540 vsize=415640 CPUtime=0.07 cores=1,3,5,7
/proc/6578/stat : 6578 (python) S 6540 6540 2614 0 -1 4202560 914 0 0 0 4 3 0 0 20 0 1 0 338255763 425615360 20226 33554432000 4194304 5683644 140733316784832 140733316775920 240565191139 0 0 16781312 33554946 18446744071580425449 0 0 17 1 0 0 0 0 0
/proc/6578/statm: 103910 20226 358 364 0 39130 0
[pid=6580] ppid=6578 vsize=801068 CPUtime=46.87 cores=1,3,5,7
/proc/6580/stat : 6580 (pbct-0.1.2-linu) R 6578 6580 6580 0 -1 4202496 193699 0 0 0 4624 63 0 0 20 0 1 0 338255764 820293632 196311 33554432000 134512640 136422188 4286627808 4286625532 135874785 0 0 16781312 16387 0 0 0 17 5 0 0 0 0 0
/proc/6580/statm: 200267 196311 215 467 0 199798 0
[pid=6540/tid=6547] ppid=6538 vsize=491420 CPUtime=0 cores=1,3,5,7
/proc/6540/task/6547/stat : 6547 (python) S 6538 6540 2614 0 -1 4202560 26 1012 0 0 0 0 0 0 20 0 3 0 338250729 503214080 21779 33554432000 4194304 5683644 140733316784832 139730479198768 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6540/tid=6579] ppid=6538 vsize=491420 CPUtime=0 cores=1,3,5,7
/proc/6540/task/6579/stat : 6579 (python) S 6538 6540 2614 0 -1 4202560 21 1012 0 0 0 0 0 0 20 0 3 0 338255764 503214080 21779 33554432000 4194304 5683644 140733316784832 139730468708912 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 101.58
Current children cumulated vsize (KiB) 2186336

[startup+162.301 s]
/proc/loadavg: 2.37 3.41 3.11 3/201 6593
/proc/meminfo: memFree=23074032/32873844 swapFree=7116/7116
[pid=6540] ppid=6538 vsize=567216 CPUtime=14.05 cores=1,3,5,7
/proc/6540/stat : 6540 (python) R 6538 6540 2614 0 -1 4202496 30381 1012 1 0 1383 22 0 0 20 0 4 0 338250222 580829184 21788 33554432000 4194304 5683644 140733316784832 140733316779152 4562916 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/6540/statm: 141804 21788 1907 364 0 77021 0
[pid=6546] ppid=6540 vsize=415628 CPUtime=0.08 cores=1,3,5,7
/proc/6546/stat : 6546 (python) S 6540 6540 2614 0 -1 4202560 1131 0 0 0 4 4 0 0 20 0 2 0 338250729 425603072 20265 33554432000 4194304 5683644 140733316784832 140733316776480 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/6546/statm: 103907 20265 396 364 0 39130 0
[pid=6548] ppid=6546 vsize=62580 CPUtime=50.29 cores=1,3,5,7
/proc/6548/stat : 6548 (wbo1.4b-fixed) T 6546 6548 6548 0 -1 4202496 17485 0 0 0 4954 75 0 0 20 0 1 0 338250729 64081920 15016 33554432000 134512640 135752315 4291332368 4291330368 134611404 0 0 16781312 3 18446744071579387992 0 0 17 5 0 0 0 0 0
/proc/6548/statm: 15645 15016 146 303 0 15340 0
[pid=6546/tid=6577] ppid=6540 vsize=415628 CPUtime=0 cores=1,3,5,7
/proc/6546/task/6577/stat : 6577 (python) S 6540 6540 2614 0 -1 4202560 43 0 0 0 0 0 0 0 20 0 2 0 338255761 425603072 20265 33554432000 4194304 5683644 140733316784832 139730479198768 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6578] ppid=6540 vsize=415640 CPUtime=0.08 cores=1,3,5,7
/proc/6578/stat : 6578 (python) S 6540 6540 2614 0 -1 4202560 1071 0 0 0 4 4 0 0 20 0 2 0 338255763 425615360 20252 33554432000 4194304 5683644 140733316784832 140733316776480 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/6578/statm: 103910 20252 383 364 0 39130 0
[pid=6580] ppid=6578 vsize=801200 CPUtime=47.64 cores=1,3,5,7
/proc/6580/stat : 6580 (pbct-0.1.2-linu) T 6578 6580 6580 0 -1 4202496 193726 0 0 0 4700 64 0 0 20 0 1 0 338255764 820428800 196338 33554432000 134512640 136422188 4286627808 4286625424 134862210 0 0 16781312 16387 18446744071579387992 0 0 17 5 0 0 0 0 0
/proc/6580/statm: 200300 196338 215 467 0 199831 0
[pid=6578/tid=6585] ppid=6540 vsize=415640 CPUtime=0 cores=1,3,5,7
/proc/6578/task/6585/stat : 6585 (python) S 6540 6540 2614 0 -1 4202560 47 0 0 0 0 0 0 0 20 0 2 0 338260530 425615360 20252 33554432000 4194304 5683644 140733316784832 139730479198768 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0

################
# More data... #
################

[pid=6685/tid=6691] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6691/stat : 6691 (java) S 6683 6685 6685 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338280963 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689589033896 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=6685/tid=6692] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6692/stat : 6692 (java) S 6683 6685 6685 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338280963 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689587981352 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=6685/tid=6693] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6693/stat : 6693 (java) S 6683 6685 6685 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338280963 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689586928296 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=6685/tid=6694] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6694/stat : 6694 (java) S 6683 6685 6685 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 18 0 338280963 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689585875752 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=6685/tid=6695] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6695/stat : 6695 (java) S 6683 6685 6685 0 -1 4202560 2 0 0 0 0 0 0 0 20 0 18 0 338280965 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689455180136 240568547241 0 0 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=6685/tid=6696] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6696/stat : 6696 (java) S 6683 6685 6685 0 -1 4202560 16 0 0 0 0 0 0 0 20 0 18 0 338280965 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689454126456 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=6685/tid=6697] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6697/stat : 6697 (java) S 6683 6685 6685 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 18 0 338280965 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689453073288 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=6685/tid=6698] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6698/stat : 6698 (java) S 6683 6685 6685 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338280968 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689452022032 240568555296 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=6685/tid=6699] ppid=6683 vsize=9558408 CPUtime=0.26 cores=1,3,5,7
/proc/6685/task/6699/stat : 6699 (java) S 6683 6685 6685 0 -1 4202560 2330 0 0 0 26 0 0 0 20 0 18 0 338280968 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689450969912 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=6685/tid=6700] ppid=6683 vsize=9558408 CPUtime=0.22 cores=1,3,5,7
/proc/6685/task/6700/stat : 6700 (java) S 6683 6685 6685 0 -1 4202560 2660 0 0 0 22 0 0 0 20 0 18 0 338280968 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689449917368 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=6685/tid=6701] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6701/stat : 6701 (java) S 6683 6685 6685 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338280968 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689448864504 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=6685/tid=6702] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6702/stat : 6702 (java) S 6683 6685 6685 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338280968 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689447812168 240568547241 0 4 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=6540/tid=6547] ppid=6538 vsize=870380 CPUtime=0 cores=1,3,5,7
/proc/6540/task/6547/stat : 6547 (python) S 6538 6540 2614 0 -1 4202560 26 1012 0 0 0 0 0 0 20 0 8 0 338250729 891269120 21819 33554432000 4194304 5683644 140733316784832 139730479198768 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6540/tid=6579] ppid=6538 vsize=870380 CPUtime=0 cores=1,3,5,7
/proc/6540/task/6579/stat : 6579 (python) S 6538 6540 2614 0 -1 4202560 21 1012 0 0 0 0 0 0 20 0 8 0 338255764 891269120 21819 33554432000 4194304 5683644 140733316784832 139730468708912 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=6540/tid=6587] ppid=6538 vsize=870380 CPUtime=0 cores=1,3,5,7
/proc/6540/task/6587/stat : 6587 (python) S 6538 6540 2614 0 -1 4202560 21 1012 0 0 0 0 0 0 20 0 8 0 338260533 891269120 21819 33554432000 4194304 5683644 140733316784832 139730458219056 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6540/tid=6595] ppid=6538 vsize=870380 CPUtime=0 cores=1,3,5,7
/proc/6540/task/6595/stat : 6595 (python) S 6538 6540 2614 0 -1 4202560 22 1012 0 0 0 0 0 0 20 0 8 0 338266686 891269120 21819 33554432000 4194304 5683644 140733316784832 139730447729200 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6540/tid=6621] ppid=6538 vsize=870380 CPUtime=0 cores=1,3,5,7
/proc/6540/task/6621/stat : 6621 (python) S 6538 6540 2614 0 -1 4202560 21 1012 0 0 0 0 0 0 20 0 8 0 338271453 891269120 21819 33554432000 4194304 5683644 140733316784832 139730437239344 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6540/tid=6676] ppid=6538 vsize=870380 CPUtime=0 cores=1,3,5,7
/proc/6540/task/6676/stat : 6676 (python) S 6538 6540 2614 0 -1 4202560 22 1012 0 0 0 0 0 0 20 0 8 0 338276244 891269120 21819 33554432000 4194304 5683644 140733316784832 139730418938416 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=6540/tid=6684] ppid=6538 vsize=870380 CPUtime=0 cores=1,3,5,7
/proc/6540/task/6684/stat : 6684 (python) S 6538 6540 2614 0 -1 4202560 22 1012 0 0 0 0 0 0 20 0 8 0 338280961 891269120 21819 33554432000 4194304 5683644 140733316784832 139730408448560 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 307.86
Current children cumulated vsize (KiB) 19119800

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+307.901 s]
/proc/loadavg: 2.14 2.91 2.96 3/228 6702
/proc/meminfo: memFree=25315396/32873844 swapFree=7116/7116
[pid=6540] ppid=6538 vsize=870380 CPUtime=16.48 cores=1,3,5,7
/proc/6540/stat : 6540 (python) S 6538 6540 2614 0 -1 4202496 33389 1012 1 0 1623 25 0 0 20 0 8 0 338250222 891269120 21819 33554432000 4194304 5683644 140733316784832 140733316779664 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/6540/statm: 217595 21819 1915 364 0 152800 0
[pid=6546] ppid=6540 vsize=415628 CPUtime=0.08 cores=1,3,5,7
/proc/6546/stat : 6546 (python) S 6540 6540 2614 0 -1 4202560 1131 0 0 0 4 4 0 0 20 0 2 0 338250729 425603072 20265 33554432000 4194304 5683644 140733316784832 140733316776480 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/6546/statm: 103907 20265 396 364 0 39130 0
[pid=6548] ppid=6546 vsize=62580 CPUtime=50.29 cores=1,3,5,7
/proc/6548/stat : 6548 (wbo1.4b-fixed) T 6546 6548 6548 0 -1 4202496 17485 0 0 0 4954 75 0 0 20 0 1 0 338250729 64081920 15016 33554432000 134512640 135752315 4291332368 4291330368 134611404 0 0 16781312 3 18446744071579387992 0 0 17 5 0 0 0 0 0
/proc/6548/statm: 15645 15016 146 303 0 15340 0
[pid=6546/tid=6577] ppid=6540 vsize=415628 CPUtime=0 cores=1,3,5,7
/proc/6546/task/6577/stat : 6577 (python) S 6540 6540 2614 0 -1 4202560 43 0 0 0 0 0 0 0 20 0 2 0 338255761 425603072 20265 33554432000 4194304 5683644 140733316784832 139730479198768 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6578] ppid=6540 vsize=415640 CPUtime=0.08 cores=1,3,5,7
/proc/6578/stat : 6578 (python) S 6540 6540 2614 0 -1 4202560 1071 0 0 0 4 4 0 0 20 0 2 0 338255763 425615360 20252 33554432000 4194304 5683644 140733316784832 140733316776480 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/6578/statm: 103910 20252 383 364 0 39130 0
[pid=6580] ppid=6578 vsize=801200 CPUtime=47.64 cores=1,3,5,7
/proc/6580/stat : 6580 (pbct-0.1.2-linu) T 6578 6580 6580 0 -1 4202496 193726 0 0 0 4700 64 0 0 20 0 1 0 338255764 820428800 196349 33554432000 134512640 136422188 4286627808 4286625424 134862210 0 0 16781312 16387 18446744071579387992 0 0 17 5 0 0 0 0 0
/proc/6580/statm: 200300 196349 215 467 0 199831 0
[pid=6578/tid=6585] ppid=6540 vsize=415640 CPUtime=0 cores=1,3,5,7
/proc/6578/task/6585/stat : 6585 (python) S 6540 6540 2614 0 -1 4202560 47 0 0 0 0 0 0 0 20 0 2 0 338260530 425615360 20252 33554432000 4194304 5683644 140733316784832 139730479198768 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6586] ppid=6540 vsize=491432 CPUtime=0.09 cores=1,3,5,7
/proc/6586/stat : 6586 (python) S 6540 6540 2614 0 -1 4202560 1086 0 0 0 5 4 0 0 20 0 2 0 338260532 503226368 20258 33554432000 4194304 5683644 140733316784832 140733316776480 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/6586/statm: 122858 20258 383 364 0 58075 0
[pid=6588] ppid=6586 vsize=58920 CPUtime=49.41 cores=1,3,5,7
/proc/6588/stat : 6588 (wbo1.4a) T 6586 6588 6588 0 -1 4202496 16591 0 0 0 4861 80 0 0 20 0 1 0 338260533 60334080 14053 33554432000 134512640 135751879 4287771904 4287770080 134616876 0 0 16781312 3 18446744071579387992 0 0 17 5 0 0 0 0 0
/proc/6588/statm: 14730 14053 143 303 0 14425 0
[pid=6586/tid=6593] ppid=6540 vsize=491432 CPUtime=0 cores=1,3,5,7
/proc/6586/task/6593/stat : 6593 (python) S 6540 6540 2614 0 -1 4202560 46 0 0 0 0 0 0 0 20 0 2 0 338265476 503226368 20258 33554432000 4194304 5683644 140733316784832 139730468708912 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6594] ppid=6540 vsize=567224 CPUtime=0.07 cores=1,3,5,7
/proc/6594/stat : 6594 (python) S 6540 6540 2614 0 -1 4202560 1086 0 0 0 5 2 0 0 20 0 2 0 338266686 580837376 20264 33554432000 4194304 5683644 140733316784832 140733316776480 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/6594/statm: 141806 20264 383 364 0 77020 0
[pid=6596] ppid=6594 vsize=3594292 CPUtime=47.61 cores=1,3,5,7
/proc/6596/stat : 6596 (minion) T 6594 6596 6596 0 -1 4202496 888284 0 0 0 4628 133 0 0 20 0 1 0 338266686 3680555008 891163 33554432000 134512640 165565024 4287115184 4287112432 142764899 0 0 16781312 8396802 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/6596/statm: 898573 891163 358 7582 0 890988 0
[pid=6594/tid=6619] ppid=6540 vsize=567224 CPUtime=0 cores=1,3,5,7
/proc/6594/task/6619/stat : 6619 (python) S 6540 6540 2614 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 338271450 580837376 20264 33554432000 4194304 5683644 140733316784832 139730458219056 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=6620] ppid=6540 vsize=643016 CPUtime=0.07 cores=1,3,5,7
/proc/6620/stat : 6620 (python) S 6540 6540 2614 0 -1 4202560 1108 0 0 0 4 3 0 0 20 0 2 0 338271453 658448384 20270 33554432000 4194304 5683644 140733316784832 140733316776480 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/6620/statm: 160754 20270 383 364 0 95965 0
[pid=6622] ppid=6620 vsize=84320 CPUtime=47.86 cores=1,3,5,7
/proc/6622/stat : 6622 (bsolo_pb10) T 6620 6622 6622 0 -1 4202496 21232 0 0 0 4726 60 0 0 20 0 1 0 338271453 86343680 18632 33554432000 134512640 137138936 4287163984 4287162912 134542448 0 0 16781312 16384 18446744071579387992 0 0 17 5 0 0 0 0 0
/proc/6622/statm: 21080 18632 192 642 0 20436 0
[pid=6620/tid=6674] ppid=6540 vsize=643016 CPUtime=0 cores=1,3,5,7
/proc/6620/task/6674/stat : 6674 (python) S 6540 6540 2614 0 -1 4202560 43 0 0 0 0 0 0 0 20 0 2 0 338276241 658448384 20270 33554432000 4194304 5683644 140733316784832 139730447729200 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6675] ppid=6540 vsize=718808 CPUtime=0.08 cores=1,3,5,7
/proc/6675/stat : 6675 (python) S 6540 6540 2614 0 -1 4202560 1101 0 0 0 5 3 0 0 20 0 2 0 338276244 736059392 20276 33554432000 4194304 5683644 140733316784832 140733316776480 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/6675/statm: 179702 20276 383 364 0 114910 0
[pid=6677] ppid=6675 vsize=43352 CPUtime=47.09 cores=1,3,5,7
/proc/6677/stat : 6677 (clasp-1.3.7-x86) T 6675 6677 6677 0 -1 4202496 11544 0 0 0 4706 3 0 0 20 0 1 0 338276244 44392448 10402 33554432000 134512640 136139019 4293157264 4293156240 134634749 0 0 16781312 16386 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/6677/statm: 10838 10402 183 398 0 10438 0
[pid=6675/tid=6682] ppid=6540 vsize=718808 CPUtime=0 cores=1,3,5,7
/proc/6675/task/6682/stat : 6682 (python) S 6540 6540 2614 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 338280958 736059392 20276 33554432000 4194304 5683644 140733316784832 139730437239344 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=6683] ppid=6540 vsize=794600 CPUtime=0 cores=1,3,5,7
/proc/6683/stat : 6683 (python) S 6540 6540 2614 0 -1 4202560 830 0 0 0 0 0 0 0 20 0 1 0 338280960 813670400 20233 33554432000 4194304 5683644 140733316784832 140733316775920 240565191139 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/6683/statm: 198650 20233 335 364 0 133855 0
[pid=6685] ppid=6683 vsize=9558408 CPUtime=1.01 cores=1,3,5,7
/proc/6685/stat : 6685 (java) S 6683 6685 6685 0 -1 4202496 8874 0 1 0 97 4 0 0 20 0 18 0 338280961 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140734032063168 240568533053 0 0 0 16800975 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/6685/statm: 2389602 26219 2230 9 0 2380296 0
[pid=6685/tid=6686] ppid=6683 vsize=9558408 CPUtime=0.49 cores=1,3,5,7
/proc/6685/task/6686/stat : 6686 (java) R 6683 6685 6685 0 -1 4202560 2464 0 1 0 47 2 0 0 20 0 18 0 338280962 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689631748728 140689455626888 0 4 0 16800975 0 0 0 -1 5 0 0 0 0 0
[pid=6685/tid=6687] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6687/stat : 6687 (java) S 6683 6685 6685 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338280963 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689593244072 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=6685/tid=6688] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6688/stat : 6688 (java) S 6683 6685 6685 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338280963 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689592191528 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=6685/tid=6689] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6689/stat : 6689 (java) S 6683 6685 6685 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338280963 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689591138472 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6685/tid=6690] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6690/stat : 6690 (java) S 6683 6685 6685 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338280963 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689590085928 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=6685/tid=6691] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6691/stat : 6691 (java) S 6683 6685 6685 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338280963 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689589033896 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=6685/tid=6692] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6692/stat : 6692 (java) S 6683 6685 6685 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338280963 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689587981352 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=6685/tid=6693] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6693/stat : 6693 (java) S 6683 6685 6685 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338280963 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689586928296 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=6685/tid=6694] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6694/stat : 6694 (java) S 6683 6685 6685 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 18 0 338280963 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689585875752 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=6685/tid=6695] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6695/stat : 6695 (java) S 6683 6685 6685 0 -1 4202560 2 0 0 0 0 0 0 0 20 0 18 0 338280965 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689455180136 240568547241 0 0 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=6685/tid=6696] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6696/stat : 6696 (java) S 6683 6685 6685 0 -1 4202560 16 0 0 0 0 0 0 0 20 0 18 0 338280965 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689454126456 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=6685/tid=6697] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6697/stat : 6697 (java) S 6683 6685 6685 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 18 0 338280965 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689453073288 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=6685/tid=6698] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6698/stat : 6698 (java) S 6683 6685 6685 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338280968 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689452022032 240568555296 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=6685/tid=6699] ppid=6683 vsize=9558408 CPUtime=0.26 cores=1,3,5,7
/proc/6685/task/6699/stat : 6699 (java) S 6683 6685 6685 0 -1 4202560 2330 0 0 0 26 0 0 0 20 0 18 0 338280968 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689450969912 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=6685/tid=6700] ppid=6683 vsize=9558408 CPUtime=0.22 cores=1,3,5,7
/proc/6685/task/6700/stat : 6700 (java) S 6683 6685 6685 0 -1 4202560 2660 0 0 0 22 0 0 0 20 0 18 0 338280968 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689449917368 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=6685/tid=6701] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6701/stat : 6701 (java) S 6683 6685 6685 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338280968 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689448864504 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=6685/tid=6702] ppid=6683 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/6685/task/6702/stat : 6702 (java) S 6683 6685 6685 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338280968 9787809792 26219 33554432000 1073741824 1073778376 140734032072032 140689447812168 240568547241 0 4 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=6540/tid=6547] ppid=6538 vsize=870380 CPUtime=0 cores=1,3,5,7
/proc/6540/task/6547/stat : 6547 (python) S 6538 6540 2614 0 -1 4202560 26 1012 0 0 0 0 0 0 20 0 8 0 338250729 891269120 21819 33554432000 4194304 5683644 140733316784832 139730479198768 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6540/tid=6579] ppid=6538 vsize=870380 CPUtime=0 cores=1,3,5,7
/proc/6540/task/6579/stat : 6579 (python) S 6538 6540 2614 0 -1 4202560 21 1012 0 0 0 0 0 0 20 0 8 0 338255764 891269120 21819 33554432000 4194304 5683644 140733316784832 139730468708912 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=6540/tid=6587] ppid=6538 vsize=870380 CPUtime=0 cores=1,3,5,7
/proc/6540/task/6587/stat : 6587 (python) S 6538 6540 2614 0 -1 4202560 21 1012 0 0 0 0 0 0 20 0 8 0 338260533 891269120 21819 33554432000 4194304 5683644 140733316784832 139730458219056 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6540/tid=6595] ppid=6538 vsize=870380 CPUtime=0 cores=1,3,5,7
/proc/6540/task/6595/stat : 6595 (python) S 6538 6540 2614 0 -1 4202560 22 1012 0 0 0 0 0 0 20 0 8 0 338266686 891269120 21819 33554432000 4194304 5683644 140733316784832 139730447729200 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6540/tid=6621] ppid=6538 vsize=870380 CPUtime=0 cores=1,3,5,7
/proc/6540/task/6621/stat : 6621 (python) S 6538 6540 2614 0 -1 4202560 21 1012 0 0 0 0 0 0 20 0 8 0 338271453 891269120 21819 33554432000 4194304 5683644 140733316784832 139730437239344 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=6540/tid=6676] ppid=6538 vsize=870380 CPUtime=0 cores=1,3,5,7
/proc/6540/task/6676/stat : 6676 (python) S 6538 6540 2614 0 -1 4202560 22 1012 0 0 0 0 0 0 20 0 8 0 338276244 891269120 21819 33554432000 4194304 5683644 140733316784832 139730418938416 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=6540/tid=6684] ppid=6538 vsize=870380 CPUtime=0 cores=1,3,5,7
/proc/6540/task/6684/stat : 6684 (python) S 6538 6540 2614 0 -1 4202560 22 1012 0 0 0 0 0 0 20 0 8 0 338280961 891269120 21819 33554432000 4194304 5683644 140733316784832 139730408448560 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 307.86
Current children cumulated vsize (KiB) 19119800

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 6540 and gives
#  childrusage.ru_utime.tv_sec=16
#  childrusage.ru_utime.tv_usec=237531
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=250961
# CPU time returned by wait4() is 16.4885
# while last known CPU time is 307.86
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 307.963
CPU time (s): 307.86
CPU user time (s): 303.22
CPU system time (s): 4.64
CPU usage (%): 99.9665
Max. virtual memory (cumulated for all children) (KiB): 19119800

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 16.2375
system time used= 0.250961
maximum resident set size= 95456
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 34401
page faults= 1
swaps= 0
block input operations= 0
block output operations= 25384
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2064
involuntary context switches= 37

runsolver used 0.98285 second user time and 2.73558 second system time

The end

Launcher Data 

Begin job on node146 at 2012-06-03 13:00:50
IDJOB=3741124
IDBENCH=89784
IDSOLVER=2327
FILE ID=node146/3741124-1338721247
RUNJOBID= node146-1338713700-2631
PBS_JOBID= 14636739
Free space on /tmp= 66708 MiB

SOLVER NAME= PB11: borg pb-dec-11.04.03
BENCH NAME= PB11/normalized-PB11/DEC-SMALLINT-LIN/lopes/normalized-31.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.dec.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3741124-1338721247/watcher-3741124-1338721247 -o /tmp/evaluation-result-3741124-1338721247/solver-3741124-1338721247 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3741124-1338721247.opb -seed 1895152942 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 1231d9e508dc821e38b22e6db3fd2b2f
RANDOM SEED=1895152942

node146.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        18073360 kB
Buffers:          258840 kB
Cached:          8931044 kB
SwapCached:         1792 kB
Active:          5003468 kB
Inactive:        8548624 kB
Active(anon):    4363096 kB
Inactive(anon):     1688 kB
Active(file):     640372 kB
Inactive(file):  8546936 kB
Unevictable:        7116 kB
Mlocked:            7116 kB
SwapTotal:      67108856 kB
SwapFree:       67099888 kB
Dirty:           3715308 kB
Writeback:             0 kB
AnonPages:       4368560 kB
Mapped:            16748 kB
Shmem:                24 kB
Slab:            1094892 kB
SReclaimable:     529916 kB
SUnreclaim:       564976 kB
KernelStack:        1480 kB
PageTables:        14188 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:    4697896 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:   1923072 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70640 MiB
End job on node146 at 2012-06-03 13:06:03