Trace number 3482429

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg pb-dec-11.04.03? (MO) 1640.75 1634.39

General information on the benchmark

Namenormalized-PB06/SATUNSAT-SMALLINT/web/uclid_pb_benchmarks/
normalized-cache-ibm-q-full.all.ucl.opb
MD5SUM14c93c93dc53353f81a889ada18ba601
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmarkUNSAT
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark850.051
Has Objective FunctionNO
SatisfiableNO
(Un)Satisfiability was provedYES
Best value of the objective function
Optimality of the best value was proved
Number of variables81558
Total number of constraints240469
Number of constraints which are clauses235865
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints4604
Minimum length of a constraint1
Maximum length of a constraint13
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 68
Number of bits of the biggest number in a constraint 7
Biggest sum of numbers in a constraint 257
Number of bits of the biggest sum of numbers9
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.31/0.48	c INFO: loaded portfolio from HOME/etc/borg-mix+class.dec.pickle
0.31/0.48	c INFO: solving HOME/instance-3482429-1307576525.opb
1.60/1.70	c INFO: parsing took 1.20 s
3.00/4.09	c INFO: features took 2.37 CPU seconds
3.00/4.11	c INFO: running clasp-1.3.7@0 for 45 with 1796 remaining (b = 0.13)
64.05/64.00	c INFO: wrote minion input file to HOME/tmplaDAvo.minion
64.05/64.00	c INFO: running minion-0.12@0 for 45 with 1747 remaining (b = 0.04)
110.89/110.76	c INFO: running scip-2.0.1-spx@0 for 45 with 1689 remaining (b = 0.03)
159.82/159.43	c INFO: running scip-2.0.1-clp@0 for 45 with 1642 remaining (b = 0.02)
210.71/210.13	c INFO: running clasp-1.3.7@48 for 45 with 1594 remaining (b = 0.01)
257.08/256.34	c INFO: running wbo1.4a@0 for 45 with 1547 remaining (b = 0.01)
307.47/306.57	c INFO: running scip-2.0.1-clp@48 for 45 with 1497 remaining (b = 0.01)
351.29/350.11	c INFO: running clasp-1.3.7@94 for 136 with 1454 remaining (b = 0.03)
488.34/486.66	c INFO: running wbo1.4b-fixed@0 for 45 with 1317 remaining (b = 0.01)
534.42/532.57	c INFO: running scip-2.0.1-spx@46 for 45 with 1272 remaining (b = 0.01)
580.35/578.32	c INFO: running scip-2.0.1-spx@92 for 456 with 1226 remaining (b = 0.07)
1042.74/1038.95	c INFO: running bsolo_pb10-l1@0 for 91 with 770 remaining (b = 0.01)
1135.11/1130.94	c INFO: running scip-2.0.1-spx@548 for 501 with 678 remaining (b = 0.06)
1640.28/1634.08	c INFO: running sat4j-pb-v20101225@0 for 45 with 176 remaining (b = 0.00)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3482429-1307576525/watcher-3482429-1307576525 -o /tmp/evaluation-result-3482429-1307576525/solver-3482429-1307576525 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3482429-1307576525.opb -seed 923846817 -budget 1800 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.75 0.72 0.27 2/467 12691
/proc/meminfo: memFree=30090484/32950928 swapFree=45617076/67111528
[pid=12691] ppid=12689 vsize=15776 CPUtime=0 cores=1,3,5,7
/proc/12691/stat : 12691 (python) R 12689 12691 11775 0 -1 4202496 888 0 0 0 0 0 0 0 22 0 1 0 869252387 16154624 769 33554432000 4194304 5683644 140734567911392 18446744073709551615 248089359045 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/12691/statm: 3944 770 343 364 0 431 0

[startup+0.052344 s]
/proc/loadavg: 1.75 0.72 0.27 2/467 12691
/proc/meminfo: memFree=30090484/32950928 swapFree=45617076/67111528
[pid=12691] ppid=12689 vsize=29788 CPUtime=0.03 cores=1,3,5,7
/proc/12691/stat : 12691 (python) S 12689 12691 11775 0 -1 4202496 1462 0 0 0 3 0 0 0 23 0 1 0 869252387 30502912 1301 33554432000 4194304 5683644 140734567911392 18446744073709551615 248089696256 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 0
/proc/12691/statm: 7447 1301 443 364 0 817 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 29788

[startup+0.105234 s]
/proc/loadavg: 1.75 0.72 0.27 2/467 12691
/proc/meminfo: memFree=30090484/32950928 swapFree=45617076/67111528
[pid=12691] ppid=12689 vsize=96128 CPUtime=0.07 cores=1,3,5,7
/proc/12691/stat : 12691 (python) R 12689 12691 11775 0 -1 4202496 3170 929 1 0 5 2 0 0 18 0 1 0 869252387 98435072 2484 33554432000 4194304 5683644 140734567911392 18446744073709551615 5170248 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/12691/statm: 24032 2484 792 364 0 1627 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 96128

[startup+0.301156 s]
/proc/loadavg: 1.75 0.72 0.27 2/467 12691
/proc/meminfo: memFree=30090484/32950928 swapFree=45617076/67111528
[pid=12691] ppid=12689 vsize=193296 CPUtime=0.22 cores=1,3,5,7
/proc/12691/stat : 12691 (python) S 12689 12691 11775 0 -1 4202496 6575 929 1 0 17 5 0 0 17 0 1 0 869252387 197935104 4867 33554432000 4194304 5683644 140734567911392 18446744073709551615 248089696256 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 0
/proc/12691/statm: 48324 4867 1417 364 0 3677 0
Current children cumulated CPU time (s) 0.22
Current children cumulated vsize (KiB) 193296

[startup+0.700079 s]
/proc/loadavg: 1.75 0.72 0.27 2/467 12691
/proc/meminfo: memFree=30090484/32950928 swapFree=45617076/67111528
[pid=12691] ppid=12689 vsize=297628 CPUtime=0.6 cores=1,3,5,7
/proc/12691/stat : 12691 (python) R 12689 12691 11775 0 -1 4202496 12820 929 1 0 51 9 0 0 17 0 1 0 869252387 304771072 10682 33554432000 4194304 5683644 140734567911392 18446744073709551615 47080082606792 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/12691/statm: 74407 10682 1781 364 0 9300 0
Current children cumulated CPU time (s) 0.6
Current children cumulated vsize (KiB) 297628

[startup+1.501 s]
/proc/loadavg: 1.75 0.72 0.27 3/468 12696
/proc/meminfo: memFree=30042728/32950928 swapFree=45617076/67111528
[pid=12691] ppid=12689 vsize=326928 CPUtime=1.4 cores=1,3,5,7
/proc/12691/stat : 12691 (python) R 12689 12691 11775 0 -1 4202496 19788 929 1 0 130 10 0 0 19 0 1 0 869252387 334774272 17650 33554432000 4194304 5683644 140734567911392 18446744073709551615 47079916204515 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/12691/statm: 81732 17650 1781 364 0 16625 0
Current children cumulated CPU time (s) 1.4
Current children cumulated vsize (KiB) 326928

[startup+3.10065 s]
/proc/loadavg: 1.75 0.72 0.27 3/468 12696
/proc/meminfo: memFree=30018672/32950928 swapFree=45617076/67111528
[pid=12691] ppid=12689 vsize=333060 CPUtime=3 cores=1,3,5,7
/proc/12691/stat : 12691 (python) R 12689 12691 11775 0 -1 4202496 30261 929 1 0 288 12 0 0 22 0 1 0 869252387 341053440 19613 33554432000 4194304 5683644 140734567911392 18446744073709551615 47079916028656 0 0 16781312 33554434 0 0 0 17 1 0 0 0
/proc/12691/statm: 83265 19613 1816 364 0 18158 0
Current children cumulated CPU time (s) 3
Current children cumulated vsize (KiB) 333060

[startup+6.30115 s]
/proc/loadavg: 1.77 0.75 0.28 5/475 12703
/proc/meminfo: memFree=29942124/32950928 swapFree=45617076/67111528
[pid=12691] ppid=12689 vsize=347088 CPUtime=4.01 cores=1,3,5,7
/proc/12691/stat : 12691 (python) S 12689 12691 11775 0 -1 4202496 31485 929 1 0 388 13 0 0 25 0 2 0 869252387 355418112 20613 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101524939 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0
/proc/12691/statm: 86772 20613 1872 364 0 21659 0
[pid=12698] ppid=12691 vsize=336844 CPUtime=0 cores=1,3,5,7
/proc/12698/stat : 12698 (python) S 12691 12691 11775 0 -1 4202560 725 0 0 0 0 0 0 0 18 0 1 0 869252799 344928256 19088 33554432000 4194304 5683644 140734567911392 18446744073709551615 248089718287 0 0 16781312 33554946 4445470135532093700 0 0 17 7 0 0 0
/proc/12698/statm: 84211 19088 351 364 0 19098 0
[pid=12700] ppid=12698 vsize=33828 CPUtime=2.17 cores=1,3,5,7
/proc/12700/stat : 12700 (clasp-1.3.7-x86) R 12698 12700 12700 0 -1 4202496 10962 0 0 0 216 1 0 0 25 0 1 0 869252799 34639872 8098 33554432000 134512640 136139019 4287338672 18446744073709551615 134608261 0 0 16781312 16386 0 0 0 17 1 0 0 0
/proc/12700/statm: 8457 8098 184 398 0 8057 0
[pid=12691/tid=12699] ppid=12689 vsize=347088 CPUtime=0 cores=1,3,5,7
/proc/12691/task/12699/stat : 12699 (python) S 12689 12691 11775 0 -1 4202560 26 929 0 0 0 0 0 0 25 0 2 0 869252799 355418112 20613 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554434 18446604469005535936 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 6.18
Current children cumulated vsize (KiB) 717760

[startup+12.701 s]
/proc/loadavg: 1.79 0.77 0.28 5/475 12703
/proc/meminfo: memFree=29939668/32950928 swapFree=45617076/67111528
[pid=12691] ppid=12689 vsize=347088 CPUtime=4.01 cores=1,3,5,7
/proc/12691/stat : 12691 (python) S 12689 12691 11775 0 -1 4202496 31485 929 1 0 388 13 0 0 25 0 2 0 869252387 355418112 20613 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101524939 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0
/proc/12691/statm: 86772 20613 1872 364 0 21659 0
[pid=12698] ppid=12691 vsize=336844 CPUtime=0.03 cores=1,3,5,7
/proc/12698/stat : 12698 (python) S 12691 12691 11775 0 -1 4202560 889 0 0 0 1 2 0 0 18 0 1 0 869252799 344928256 19112 33554432000 4194304 5683644 140734567911392 18446744073709551615 248089718287 0 0 16781312 33554946 5623753486446373585 0 0 17 7 0 0 0
/proc/12698/statm: 84211 19112 375 364 0 19098 0
[pid=12700] ppid=12698 vsize=35476 CPUtime=8.57 cores=1,3,5,7
/proc/12700/stat : 12700 (clasp-1.3.7-x86) R 12698 12700 12700 0 -1 4202496 11357 0 0 0 855 2 0 0 25 0 1 0 869252799 36327424 8493 33554432000 134512640 136139019 4287338672 18446744073709551615 134883066 0 0 16781312 16386 0 0 0 17 1 0 0 0
/proc/12700/statm: 8869 8493 184 398 0 8469 0
[pid=12691/tid=12699] ppid=12689 vsize=347088 CPUtime=0 cores=1,3,5,7
/proc/12691/task/12699/stat : 12699 (python) S 12689 12691 11775 0 -1 4202560 26 929 0 0 0 0 0 0 25 0 2 0 869252799 355418112 20613 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554434 18446604469005535936 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 12.61
Current children cumulated vsize (KiB) 719408

[startup+25.5007 s]
/proc/loadavg: 1.84 0.83 0.31 3/471 12775
/proc/meminfo: memFree=29976120/32950928 swapFree=45617076/67111528
[pid=12691] ppid=12689 vsize=347088 CPUtime=4.01 cores=1,3,5,7
/proc/12691/stat : 12691 (python) S 12689 12691 11775 0 -1 4202496 31485 929 1 0 388 13 0 0 25 0 2 0 869252387 355418112 20613 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101524939 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0
/proc/12691/statm: 86772 20613 1872 364 0 21659 0
[pid=12698] ppid=12691 vsize=336844 CPUtime=0.07 cores=1,3,5,7
/proc/12698/stat : 12698 (python) S 12691 12691 11775 0 -1 4202560 892 0 0 0 2 5 0 0 18 0 1 0 869252799 344928256 19112 33554432000 4194304 5683644 140734567911392 18446744073709551615 248089718287 0 0 16781312 33554946 5623753486446373585 0 0 17 7 0 0 0
/proc/12698/statm: 84211 19112 375 364 0 19098 0
[pid=12700] ppid=12698 vsize=38028 CPUtime=21.37 cores=1,3,5,7
/proc/12700/stat : 12700 (clasp-1.3.7-x86) R 12698 12700 12700 0 -1 4202496 11990 0 0 0 2134 3 0 0 25 0 1 0 869252799 38940672 9126 33554432000 134512640 136139019 4287338672 18446744073709551615 134608289 0 0 16781312 16386 0 0 0 17 1 0 0 0
/proc/12700/statm: 9507 9126 184 398 0 9107 0
[pid=12691/tid=12699] ppid=12689 vsize=347088 CPUtime=0 cores=1,3,5,7
/proc/12691/task/12699/stat : 12699 (python) S 12689 12691 11775 0 -1 4202560 26 929 0 0 0 0 0 0 25 0 2 0 869252799 355418112 20613 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554434 18446604469005535936 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 25.45
Current children cumulated vsize (KiB) 721960

[startup+51.1012 s]
/proc/loadavg: 2.02 0.95 0.37 3/471 12775
/proc/meminfo: memFree=29972048/32950928 swapFree=45617076/67111528
[pid=12691] ppid=12689 vsize=347088 CPUtime=4.01 cores=1,3,5,7
/proc/12691/stat : 12691 (python) S 12689 12691 11775 0 -1 4202496 31485 929 1 0 388 13 0 0 25 0 2 0 869252387 355418112 20613 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101524939 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0
/proc/12691/statm: 86772 20613 1872 364 0 21659 0
[pid=12698] ppid=12691 vsize=336844 CPUtime=0.16 cores=1,3,5,7
/proc/12698/stat : 12698 (python) S 12691 12691 11775 0 -1 4202560 1001 0 0 0 6 10 0 0 18 0 1 0 869252799 344928256 19117 33554432000 4194304 5683644 140734567911392 18446744073709551615 248089718287 0 0 16781312 33554946 5623753486446373585 0 0 17 7 0 0 0
/proc/12698/statm: 84211 19117 380 364 0 19098 0
[pid=12700] ppid=12698 vsize=41668 CPUtime=46.96 cores=1,3,5,7
/proc/12700/stat : 12700 (clasp-1.3.7-x86) R 12698 12700 12700 0 -1 4202496 12889 0 0 0 4692 4 0 0 25 0 1 0 869252799 42668032 10025 33554432000 134512640 136139019 4287338672 18446744073709551615 134608160 0 0 16781312 16386 0 0 0 17 1 0 0 0
/proc/12700/statm: 10417 10025 184 398 0 10017 0
[pid=12691/tid=12699] ppid=12689 vsize=347088 CPUtime=0 cores=1,3,5,7
/proc/12691/task/12699/stat : 12699 (python) S 12689 12691 11775 0 -1 4202560 26 929 0 0 0 0 0 0 25 0 2 0 869252799 355418112 20613 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554434 18446604469005535936 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 51.13
Current children cumulated vsize (KiB) 725600

[startup+102.301 s]
/proc/loadavg: 2.07 1.13 0.45 3/479 12785
/proc/meminfo: memFree=29695708/32950928 swapFree=45617076/67111528
[pid=12691] ppid=12689 vsize=357344 CPUtime=15.69 cores=1,3,5,7
/proc/12691/stat : 12691 (python) S 12689 12691 11775 0 -1 4202496 32449 929 1 0 1554 15 0 0 25 0 3 0 869252387 365920256 20620 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101524939 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0
/proc/12691/statm: 89336 20620 1875 364 0 24220 0
[pid=12698] ppid=12691 vsize=347088 CPUtime=0.18 cores=1,3,5,7
/proc/12698/stat : 12698 (python) S 12691 12691 11775 0 -1 4202560 1133 0 0 0 7 11 0 0 19 0 2 0 869252799 355418112 19153 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101524939 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0
/proc/12698/statm: 86772 19153 411 364 0 21659 0
[pid=12700] ppid=12698 vsize=41824 CPUtime=48.18 cores=1,3,5,7
/proc/12700/stat : 12700 (clasp-1.3.7-x86) T 12698 12700 12700 0 -1 4202496 12919 0 0 0 4814 4 0 0 25 0 1 0 869252799 42827776 10055 33554432000 134512640 136139019 4287338672 18446744073709551615 134883094 0 0 16781312 16386 18446744071562691360 0 0 17 1 0 0 0
/proc/12700/statm: 10456 10055 184 398 0 10056 0
[pid=12698/tid=12776] ppid=12691 vsize=347088 CPUtime=0 cores=1,3,5,7
/proc/12698/task/12776/stat : 12776 (python) S 12691 12691 11775 0 -1 4202560 41 0 0 0 0 0 0 0 19 0 2 0 869257619 355418112 19153 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554946 18446604454671050752 0 0 -1 3 0 0 0
[pid=12778] ppid=12691 vsize=347100 CPUtime=0.14 cores=1,3,5,7
/proc/12778/stat : 12778 (python) S 12691 12691 11775 0 -1 4202560 953 0 0 0 7 7 0 0 15 0 1 0 869258788 355430400 19113 33554432000 4194304 5683644 140734567911392 18446744073709551615 248089718342 0 0 16781312 33554946 5623753486446373585 0 0 17 7 0 0 0
/proc/12778/statm: 86775 19113 372 364 0 21659 0
[pid=12780] ppid=12778 vsize=218784 CPUtime=38.28 cores=1,3,5,7
/proc/12780/stat : 12780 (minion) R 12778 12780 12780 0 -1 4202496 62903 0 0 0 3821 7 0 0 25 0 1 0 869258789 224034816 47147 33554432000 134512640 165565024 4293344624 18446744073709551615 142853056 0 0 16781312 8396802 0 0 0 17 3 0 0 0
/proc/12780/statm: 54696 47147 346 7582 0 47111 0
[pid=12691/tid=12699] ppid=12689 vsize=357344 CPUtime=0 cores=1,3,5,7
/proc/12691/task/12699/stat : 12699 (python) S 12689 12691 11775 0 -1 4202560 26 929 0 0 0 0 0 0 25 0 3 0 869252799 365920256 20620 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554434 18446604469005535936 0 0 -1 1 0 0 0
[pid=12691/tid=12779] ppid=12689 vsize=357344 CPUtime=0 cores=1,3,5,7
/proc/12691/task/12779/stat : 12779 (python) S 12689 12691 11775 0 -1 4202560 29 929 0 0 0 0 0 0 25 0 3 0 869258789 365920256 20620 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554434 18446604469005535936 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 102.47
Current children cumulated vsize (KiB) 1312140

[startup+162.302 s]
/proc/loadavg: 2.02 1.28 0.55 3/491 12798
/proc/meminfo: memFree=28506096/32950928 swapFree=45617076/67111528
[pid=12691] ppid=12689 vsize=377856 CPUtime=15.75 cores=1,3,5,7
/proc/12691/stat : 12691 (python) S 12689 12691 11775 0 -1 4202496 34257 929 1 0 1559 16 0 0 19 0 5 0 869252387 386924544 20632 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101524939 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0
/proc/12691/statm: 94464 20632 1879 364 0 29342 0
[pid=12698] ppid=12691 vsize=347088 CPUtime=0.18 cores=1,3,5,7
/proc/12698/stat : 12698 (python) S 12691 12691 11775 0 -1 4202560 1133 0 0 0 7 11 0 0 19 0 2 0 869252799 355418112 19153 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101524939 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0
/proc/12698/statm: 86772 19153 411 364 0 21659 0
[pid=12700] ppid=12698 vsize=41824 CPUtime=48.18 cores=1,3,5,7
/proc/12700/stat : 12700 (clasp-1.3.7-x86) T 12698 12700 12700 0 -1 4202496 12919 0 0 0 4814 4 0 0 25 0 1 0 869252799 42827776 10055 33554432000 134512640 136139019 4287338672 18446744073709551615 134883094 0 0 16781312 16386 18446744071562691360 0 0 17 1 0 0 0
/proc/12700/statm: 10456 10055 184 398 0 10056 0
[pid=12698/tid=12776] ppid=12691 vsize=347088 CPUtime=0 cores=1,3,5,7
/proc/12698/task/12776/stat : 12776 (python) S 12691 12691 11775 0 -1 4202560 41 0 0 0 0 0 0 0 19 0 2 0 869257619 355418112 19153 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554946 18446604454671050752 0 0 -1 3 0 0 0
[pid=12778] ppid=12691 vsize=347100 CPUtime=0.18 cores=1,3,5,7
/proc/12778/stat : 12778 (python) S 12691 12691 11775 0 -1 4202560 1125 0 0 0 9 9 0 0 16 0 2 0 869258788 355430400 19138 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101524939 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/12778/statm: 86775 19138 396 364 0 21659 0
[pid=12780] ppid=12778 vsize=218784 CPUtime=46.72 cores=1,3,5,7
/proc/12780/stat : 12780 (minion) T 12778 12780 12780 0 -1 4202496 62903 0 0 0 4665 7 0 0 25 0 1 0 869258789 224034816 47147 33554432000 134512640 165565024 4293344624 18446744073709551615 134641006 0 0 16781312 8396802 18446744071562691360 0 0 17 3 0 0 0
/proc/12780/statm: 54696 47147 346 7582 0 47111 0
[pid=12778/tid=12786] ppid=12691 vsize=347100 CPUtime=0 cores=1,3,5,7
/proc/12778/task/12786/stat : 12786 (python) S 12691 12691 11775 0 -1 4202560 51 0 0 0 0 0 0 0 16 0 2 0 869263461 355430400 19138 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554946 18446604468044407040 0 0 -1 7 0 0 0
[pid=12787] ppid=12691 vsize=357356 CPUtime=0.19 cores=1,3,5,7

################
# More data... #
################

[pid=12840/tid=12844] ppid=12691 vsize=388124 CPUtime=0 cores=1,3,5,7
/proc/12840/task/12844/stat : 12844 (python) S 12691 12691 11775 0 -1 4202560 45 0 0 0 0 0 0 0 16 0 2 0 869305643 397438976 19154 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554946 18446604469005534144 0 0 -1 7 0 0 0
[pid=12866] ppid=12691 vsize=398380 CPUtime=0.37 cores=1,3,5,7
/proc/12866/stat : 12866 (python) S 12691 12691 11775 0 -1 4202560 1177 0 0 0 18 19 0 0 17 0 2 0 869356283 407941120 19158 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101524939 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0
/proc/12866/statm: 99595 19158 396 364 0 34464 0
[pid=12868] ppid=12866 vsize=62736 CPUtime=91.97 cores=1,3,5,7
/proc/12868/stat : 12868 (bsolo_pb10) T 12866 12868 12868 0 -1 4202496 15307 0 0 0 9191 6 0 0 25 0 1 0 869356283 64241664 12849 33554432000 134512640 137138936 4288980944 18446744073709551615 136504212 0 0 16781312 16384 18446744071562691360 0 0 17 3 0 0 0
/proc/12868/statm: 15684 12849 184 642 0 15040 0
[pid=12866/tid=12875] ppid=12691 vsize=398380 CPUtime=0 cores=1,3,5,7
/proc/12866/task/12875/stat : 12875 (python) S 12691 12691 11775 0 -1 4202560 47 0 0 0 0 0 0 0 17 0 2 0 869365481 407941120 19158 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554946 18446604458986399936 0 0 -1 1 0 0 0
[pid=12691/tid=12699] ppid=12689 vsize=408624 CPUtime=0 cores=1,3,5,7
/proc/12691/task/12699/stat : 12699 (python) S 12689 12691 11775 0 -1 4202560 34 929 0 0 0 0 0 0 18 0 8 0 869252799 418430976 20650 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554434 18446604469005535936 0 0 -1 1 0 0 0
[pid=12691/tid=12779] ppid=12689 vsize=408624 CPUtime=0 cores=1,3,5,7
/proc/12691/task/12779/stat : 12779 (python) S 12689 12691 11775 0 -1 4202560 29 929 0 0 0 0 0 0 25 0 8 0 869258789 418430976 20650 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554434 18446604469005535936 0 0 -1 1 0 0 0
[pid=12691/tid=12788] ppid=12689 vsize=408624 CPUtime=0 cores=1,3,5,7
/proc/12691/task/12788/stat : 12788 (python) S 12689 12691 11775 0 -1 4202560 40 929 0 0 0 0 0 0 18 0 8 0 869263464 418430976 20650 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554434 18446604469005535936 0 0 -1 1 0 0 0
[pid=12691/tid=12797] ppid=12689 vsize=408624 CPUtime=0 cores=1,3,5,7
/proc/12691/task/12797/stat : 12797 (python) S 12689 12691 11775 0 -1 4202560 35 929 0 0 0 0 0 0 18 0 8 0 869268331 418430976 20650 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554434 18446604469005535936 0 0 -1 5 0 0 0
[pid=12691/tid=12808] ppid=12689 vsize=408624 CPUtime=0 cores=1,3,5,7
/proc/12691/task/12808/stat : 12808 (python) S 12689 12691 11775 0 -1 4202560 28 929 0 0 0 0 0 0 20 0 8 0 869278023 418430976 20650 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554434 18446604469005535936 0 0 -1 7 0 0 0
[pid=12691/tid=12841] ppid=12689 vsize=408624 CPUtime=0 cores=1,3,5,7
/proc/12691/task/12841/stat : 12841 (python) S 12689 12691 11775 0 -1 4202560 26 929 0 0 0 0 0 0 20 0 8 0 869301055 418430976 20650 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554434 18446604469005535936 0 0 -1 1 0 0 0
[pid=12691/tid=12867] ppid=12689 vsize=408624 CPUtime=0 cores=1,3,5,7
/proc/12691/task/12867/stat : 12867 (python) S 12689 12691 11775 0 -1 4202560 28 929 0 0 0 0 0 0 20 0 8 0 869356283 418430976 20650 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554434 18446604469005535936 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 1608.46
Current children cumulated vsize (KiB) 6968296



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1634.3 s]
/proc/loadavg: 2.15 2.04 1.71 6/563 12922
/proc/meminfo: memFree=24554612/32950928 swapFree=45617076/67111528
[pid=12691] ppid=12689 vsize=418880 CPUtime=15.95 cores=1,3,5,7
/proc/12691/stat : 12691 (python) S 12689 12691 11775 0 -1 4202496 38405 929 1 0 1577 18 0 0 20 0 9 0 869252387 428933120 20656 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101524939 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0
/proc/12691/statm: 104720 20656 1887 364 0 39586 0
[pid=12698] ppid=12691 vsize=347088 CPUtime=0.94 cores=1,3,5,7
/proc/12698/stat : 12698 (python) S 12691 12691 11775 0 -1 4202560 1199 0 0 0 44 50 0 0 18 0 2 0 869252799 355418112 19154 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101524939 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/12698/statm: 86772 19154 412 364 0 21659 0
[pid=12700] ppid=12698 vsize=56196 CPUtime=230.91 cores=1,3,5,7
/proc/12700/stat : 12700 (clasp-1.3.7-x86) T 12698 12700 12700 0 -1 4202496 16431 0 0 0 23086 5 0 0 25 0 1 0 869252799 57544704 13567 33554432000 134512640 136139019 4287338672 18446744073709551615 134883821 0 0 16781312 16386 18446744071562691360 0 0 17 1 0 0 0
/proc/12700/statm: 14049 13567 184 398 0 13649 0
[pid=12698/tid=12776] ppid=12691 vsize=347088 CPUtime=0 cores=1,3,5,7
/proc/12698/task/12776/stat : 12776 (python) S 12691 12691 11775 0 -1 4202560 43 0 0 0 0 0 0 0 18 0 2 0 869257619 355418112 19154 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554946 18446604454671050752 0 0 -1 3 0 0 0
[pid=12778] ppid=12691 vsize=347100 CPUtime=0.18 cores=1,3,5,7
/proc/12778/stat : 12778 (python) S 12691 12691 11775 0 -1 4202560 1125 0 0 0 9 9 0 0 16 0 2 0 869258788 355430400 19138 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101524939 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/12778/statm: 86775 19138 396 364 0 21659 0
[pid=12780] ppid=12778 vsize=218784 CPUtime=46.72 cores=1,3,5,7
/proc/12780/stat : 12780 (minion) T 12778 12780 12780 0 -1 4202496 62903 0 0 0 4665 7 0 0 25 0 1 0 869258789 224034816 47147 33554432000 134512640 165565024 4293344624 18446744073709551615 134641006 0 0 16781312 8396802 18446744071562691360 0 0 17 3 0 0 0
/proc/12780/statm: 54696 47147 346 7582 0 47111 0
[pid=12778/tid=12786] ppid=12691 vsize=347100 CPUtime=0 cores=1,3,5,7
/proc/12778/task/12786/stat : 12786 (python) S 12691 12691 11775 0 -1 4202560 51 0 0 0 0 0 0 0 16 0 2 0 869263461 355430400 19138 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554946 18446604468044407040 0 0 -1 7 0 0 0
[pid=12787] ppid=12691 vsize=357356 CPUtime=4.36 cores=1,3,5,7
/proc/12787/stat : 12787 (python) S 12691 12691 11775 0 -1 4202560 1188 0 0 0 206 230 0 0 15 0 2 0 869263464 365932544 19142 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101524939 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/12787/statm: 89339 19142 396 364 0 24220 0
[pid=12789] ppid=12787 vsize=2695208 CPUtime=1057.95 cores=1,3,5,7
/proc/12789/stat : 12789 (scip-2.0.1.linu) T 12787 12789 12789 0 -1 4202496 669867 0 0 0 105034 761 0 0 25 0 1 0 869263464 2759892992 661943 33554432000 4194304 13684479 140736867163632 18446744073709551615 5688624 0 0 16781312 2 18446744071562691360 0 0 17 1 0 0 0
/proc/12789/statm: 673802 661943 659 2317 0 671480 0
[pid=12787/tid=12795] ppid=12691 vsize=357356 CPUtime=0 cores=1,3,5,7
/proc/12787/task/12795/stat : 12795 (python) S 12691 12691 11775 0 -1 4202560 47 0 0 0 0 0 0 0 15 0 2 0 869268328 365932544 19142 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554946 18446604464924173504 0 0 -1 5 0 0 0
[pid=12796] ppid=12691 vsize=367612 CPUtime=0.41 cores=1,3,5,7
/proc/12796/stat : 12796 (python) S 12691 12691 11775 0 -1 4202560 1168 0 0 0 20 21 0 0 15 0 2 0 869268331 376434688 19146 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101524939 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/12796/statm: 91903 19146 396 364 0 26781 0
[pid=12798] ppid=12796 vsize=885748 CPUtime=94.18 cores=1,3,5,7
/proc/12798/stat : 12798 (scip-2.0.1.linu) T 12796 12798 12798 0 -1 4202496 221614 0 0 0 9206 212 0 0 25 0 1 0 869268331 907005952 213696 33554432000 4194304 15243407 140733681599792 18446744073709551615 5688591 0 0 16781312 2 18446744071562691360 0 0 17 3 0 0 0
/proc/12798/statm: 221437 213696 702 2698 0 218732 0
[pid=12796/tid=12805] ppid=12691 vsize=367612 CPUtime=0 cores=1,3,5,7
/proc/12796/task/12805/stat : 12805 (python) S 12691 12691 11775 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 869273399 376434688 19146 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554946 18446604454247884672 0 0 -1 7 0 0 0
[pid=12807] ppid=12691 vsize=377868 CPUtime=0.21 cores=1,3,5,7
/proc/12807/stat : 12807 (python) S 12691 12691 11775 0 -1 4202560 1149 0 0 0 9 12 0 0 17 0 2 0 869278022 386936832 19150 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101524939 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/12807/statm: 94467 19150 396 364 0 29342 0
[pid=12809] ppid=12807 vsize=43584 CPUtime=50.19 cores=1,3,5,7
/proc/12809/stat : 12809 (wbo1.4a) T 12807 12809 12809 0 -1 4202496 12485 0 0 0 5006 13 0 0 25 0 1 0 869278023 44630016 9865 33554432000 134512640 135751879 4289175344 18446744073709551615 134616336 0 0 16781312 3 18446744071562691360 0 0 17 3 0 0 0
/proc/12809/statm: 10896 9865 143 303 0 10591 0
[pid=12807/tid=12833] ppid=12691 vsize=377868 CPUtime=0 cores=1,3,5,7
/proc/12807/task/12833/stat : 12833 (python) S 12691 12691 11775 0 -1 4202560 45 0 0 0 0 0 0 0 17 0 2 0 869283043 386936832 19150 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554946 18446604470834717120 0 0 -1 7 0 0 0
[pid=12840] ppid=12691 vsize=388124 CPUtime=0.19 cores=1,3,5,7
/proc/12840/stat : 12840 (python) S 12691 12691 11775 0 -1 4202560 1133 0 0 0 9 10 0 0 16 0 2 0 869301054 397438976 19154 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101524939 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/12840/statm: 97031 19154 396 364 0 31903 0
[pid=12842] ppid=12840 vsize=78852 CPUtime=45.87 cores=1,3,5,7
/proc/12842/stat : 12842 (wbo1.4b-fixed) T 12840 12842 12842 0 -1 4202496 21253 0 0 0 4572 15 0 0 25 0 1 0 869301055 80744448 18636 33554432000 134512640 135752315 4289109616 18446744073709551615 134517408 0 0 16781312 3 18446744071562691360 0 0 17 3 0 0 0
/proc/12842/statm: 19713 18636 146 303 0 19408 0
[pid=12840/tid=12844] ppid=12691 vsize=388124 CPUtime=0 cores=1,3,5,7
/proc/12840/task/12844/stat : 12844 (python) S 12691 12691 11775 0 -1 4202560 45 0 0 0 0 0 0 0 16 0 2 0 869305643 397438976 19154 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554946 18446604469005534144 0 0 -1 7 0 0 0
[pid=12866] ppid=12691 vsize=398380 CPUtime=0.37 cores=1,3,5,7
/proc/12866/stat : 12866 (python) S 12691 12691 11775 0 -1 4202560 1177 0 0 0 18 19 0 0 17 0 2 0 869356283 407941120 19158 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101524939 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0
/proc/12866/statm: 99595 19158 396 364 0 34464 0
[pid=12868] ppid=12866 vsize=62736 CPUtime=91.97 cores=1,3,5,7
/proc/12868/stat : 12868 (bsolo_pb10) T 12866 12868 12868 0 -1 4202496 15307 0 0 0 9191 6 0 0 25 0 1 0 869356283 64241664 12849 33554432000 134512640 137138936 4288980944 18446744073709551615 136504212 0 0 16781312 16384 18446744071562691360 0 0 17 3 0 0 0
/proc/12868/statm: 15684 12849 184 642 0 15040 0
[pid=12866/tid=12875] ppid=12691 vsize=398380 CPUtime=0 cores=1,3,5,7
/proc/12866/task/12875/stat : 12875 (python) S 12691 12691 11775 0 -1 4202560 47 0 0 0 0 0 0 0 17 0 2 0 869365481 407941120 19158 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554946 18446604458986399936 0 0 -1 1 0 0 0
[pid=12903] ppid=12691 vsize=408636 CPUtime=0 cores=1,3,5,7
/proc/12903/stat : 12903 (python) S 12691 12691 11775 0 -1 4202560 835 0 0 0 0 0 0 0 17 0 1 0 869415796 418443264 19113 33554432000 4194304 5683644 140734567911392 18446744073709551615 248089718342 0 0 16781312 33554946 2677119571625771008 0 0 17 1 0 0 0
/proc/12903/statm: 102159 19113 348 364 0 37025 0
[pid=12905] ppid=12903 vsize=8598260 CPUtime=0.35 cores=1,3,5,7
/proc/12905/stat : 12905 (java) S 12903 12905 12905 0 -1 4202496 7404 0 1 0 34 1 0 0 17 0 18 0 869415796 8804618240 6788 33554432000 1073741824 1073778376 140737321848208 18446744073709551615 248101500901 0 0 0 16800975 18446744073709551615 0 0 17 5 0 0 0
/proc/12905/statm: 2149565 6788 2188 9 0 2140306 0
[pid=12905/tid=12906] ppid=12903 vsize=8598260 CPUtime=0.22 cores=1,3,5,7
/proc/12905/task/12906/stat : 12906 (java) R 12903 12905 12905 0 -1 4202560 4914 0 1 0 22 0 0 0 25 0 18 0 869415797 8804618240 7030 33554432000 1073741824 1073778376 140737321848208 18446744073709551615 46912505228572 0 4 0 16800975 0 0 0 -1 5 0 0 0
[pid=12905/tid=12907] ppid=12903 vsize=8598260 CPUtime=0 cores=1,3,5,7
/proc/12905/task/12907/stat : 12907 (java) S 12903 12905 12905 0 -1 4202560 1 0 0 0 0 0 0 0 19 0 18 0 869415798 8804618240 7030 33554432000 1073741824 1073778376 140737321848208 18446744073709551615 248101514137 0 4 0 16800975 18446604460414679936 0 0 -1 5 0 0 0
[pid=12905/tid=12908] ppid=12903 vsize=8598260 CPUtime=0 cores=1,3,5,7
/proc/12905/task/12908/stat : 12908 (java) S 12903 12905 12905 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 869415798 8804618240 7030 33554432000 1073741824 1073778376 140737321848208 18446744073709551615 248101514137 0 4 0 16800975 18446604460414679936 0 0 -1 1 0 0 0
[pid=12905/tid=12909] ppid=12903 vsize=8598260 CPUtime=0 cores=1,3,5,7
/proc/12905/task/12909/stat : 12909 (java) S 12903 12905 12905 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 869415798 8804618240 7031 33554432000 1073741824 1073778376 140737321848208 18446744073709551615 248101514137 0 4 0 16800975 18446604460414679936 0 0 -1 1 0 0 0
[pid=12905/tid=12910] ppid=12903 vsize=8598260 CPUtime=0 cores=1,3,5,7
/proc/12905/task/12910/stat : 12910 (java) S 12903 12905 12905 0 -1 4202560 1 0 0 0 0 0 0 0 21 0 18 0 869415798 8804618240 7031 33554432000 1073741824 1073778376 140737321848208 18446744073709551615 248101514137 0 4 0 16800975 18446604460414679936 0 0 -1 1 0 0 0
[pid=12905/tid=12911] ppid=12903 vsize=8598260 CPUtime=0 cores=1,3,5,7
/proc/12905/task/12911/stat : 12911 (java) S 12903 12905 12905 0 -1 4202560 1 0 0 0 0 0 0 0 21 0 18 0 869415798 8804618240 7031 33554432000 1073741824 1073778376 140737321848208 18446744073709551615 248101514137 0 4 0 16800975 18446604460414679936 0 0 -1 1 0 0 0
[pid=12905/tid=12912] ppid=12903 vsize=8598260 CPUtime=0 cores=1,3,5,7
/proc/12905/task/12912/stat : 12912 (java) S 12903 12905 12905 0 -1 4202560 1 0 0 0 0 0 0 0 21 0 18 0 869415798 8804618240 7032 33554432000 1073741824 1073778376 140737321848208 18446744073709551615 248101514137 0 4 0 16800975 18446604460414679936 0 0 -1 1 0 0 0
[pid=12905/tid=12913] ppid=12903 vsize=8598260 CPUtime=0 cores=1,3,5,7
/proc/12905/task/12913/stat : 12913 (java) S 12903 12905 12905 0 -1 4202560 1 0 0 0 0 0 0 0 21 0 18 0 869415798 8804618240 7032 33554432000 1073741824 1073778376 140737321848208 18446744073709551615 248101514137 0 4 0 16800975 18446604460414679936 0 0 -1 1 0 0 0
[pid=12905/tid=12914] ppid=12903 vsize=8598260 CPUtime=0 cores=1,3,5,7
/proc/12905/task/12914/stat : 12914 (java) S 12903 12905 12905 0 -1 4202560 1 0 0 0 0 0 0 0 21 0 18 0 869415798 8804618240 7032 33554432000 1073741824 1073778376 140737321848208 18446744073709551615 248101514137 0 4 0 16800975 18446604460414679936 0 0 -1 1 0 0 0
[pid=12905/tid=12915] ppid=12903 vsize=8598260 CPUtime=0 cores=1,3,5,7
/proc/12905/task/12915/stat : 12915 (java) S 12903 12905 12905 0 -1 4202560 2 0 0 0 0 0 0 0 23 0 18 0 869415799 8804618240 7032 33554432000 1073741824 1073778376 140737321848208 18446744073709551615 248101514752 0 0 0 16800975 18446604460414679936 0 0 -1 1 0 0 0
[pid=12905/tid=12916] ppid=12903 vsize=8598260 CPUtime=0 cores=1,3,5,7
/proc/12905/task/12916/stat : 12916 (java) S 12903 12905 12905 0 -1 4202560 16 0 0 0 0 0 0 0 23 0 18 0 869415799 8804618240 7033 33554432000 1073741824 1073778376 140737321848208 18446744073709551615 248101514137 0 4 0 16800975 18446604460414679936 0 0 -1 5 0 0 0
[pid=12905/tid=12917] ppid=12903 vsize=8598260 CPUtime=0 cores=1,3,5,7
/proc/12905/task/12917/stat : 12917 (java) S 12903 12905 12905 0 -1 4202560 7 0 0 0 0 0 0 0 24 0 18 0 869415799 8804618240 7033 33554432000 1073741824 1073778376 140737321848208 18446744073709551615 248101514137 0 4 0 16800975 18446604460414679936 0 0 -1 5 0 0 0
[pid=12905/tid=12918] ppid=12903 vsize=8598260 CPUtime=0 cores=1,3,5,7
/proc/12905/task/12918/stat : 12918 (java) S 12903 12905 12905 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 18 0 869415802 8804618240 7033 33554432000 1073741824 1073778376 140737321848208 18446744073709551615 248101521841 0 4 0 16800975 18446604460414679936 0 0 -1 5 0 0 0
[pid=12905/tid=12919] ppid=12903 vsize=8598260 CPUtime=0.09 cores=1,3,5,7
/proc/12905/task/12919/stat : 12919 (java) R 12903 12905 12905 0 -1 4202560 652 0 0 0 9 0 0 0 25 0 18 0 869415802 8804618240 7033 33554432000 1073741824 1073778376 140737321848208 18446744073709551615 248101514137 0 4 0 16800975 0 0 0 -1 1 0 0 0
[pid=12905/tid=12920] ppid=12903 vsize=8598260 CPUtime=0.03 cores=1,3,5,7
/proc/12905/task/12920/stat : 12920 (java) R 12903 12905 12905 0 -1 4202560 735 0 0 0 3 0 0 0 24 0 18 0 869415802 8804618240 7034 33554432000 1073741824 1073778376 140737321848208 18446744073709551615 47090868100911 0 4 0 16800975 0 0 0 -1 1 0 0 0
[pid=12905/tid=12921] ppid=12903 vsize=8598260 CPUtime=0 cores=1,3,5,7
/proc/12905/task/12921/stat : 12921 (java) S 12903 12905 12905 0 -1 4202560 2 0 0 0 0 0 0 0 25 0 18 0 869415802 8804618240 7034 33554432000 1073741824 1073778376 140737321848208 18446744073709551615 248101514137 0 4 0 16800975 18446604460414679936 0 0 -1 1 0 0 0
[pid=12905/tid=12922] ppid=12903 vsize=8598260 CPUtime=0 cores=1,3,5,7
/proc/12905/task/12922/stat : 12922 (java) S 12903 12905 12905 0 -1 4202560 1 0 0 0 0 0 0 0 24 0 18 0 869415802 8804618240 7034 33554432000 1073741824 1073778376 140737321848208 18446744073709551615 248101514752 0 4 0 16800975 18446604460414679936 0 0 -1 1 0 0 0
[pid=12691/tid=12699] ppid=12689 vsize=418880 CPUtime=0 cores=1,3,5,7
/proc/12691/task/12699/stat : 12699 (python) S 12689 12691 11775 0 -1 4202560 34 929 0 0 0 0 0 0 18 0 9 0 869252799 428933120 20656 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554434 18446604469005535936 0 0 -1 1 0 0 0
[pid=12691/tid=12779] ppid=12689 vsize=418880 CPUtime=0 cores=1,3,5,7
/proc/12691/task/12779/stat : 12779 (python) S 12689 12691 11775 0 -1 4202560 29 929 0 0 0 0 0 0 25 0 9 0 869258789 428933120 20656 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554434 18446604469005535936 0 0 -1 1 0 0 0
[pid=12691/tid=12788] ppid=12689 vsize=418880 CPUtime=0 cores=1,3,5,7
/proc/12691/task/12788/stat : 12788 (python) S 12689 12691 11775 0 -1 4202560 40 929 0 0 0 0 0 0 18 0 9 0 869263464 428933120 20656 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554434 18446604469005535936 0 0 -1 1 0 0 0
[pid=12691/tid=12797] ppid=12689 vsize=418880 CPUtime=0 cores=1,3,5,7
/proc/12691/task/12797/stat : 12797 (python) S 12689 12691 11775 0 -1 4202560 35 929 0 0 0 0 0 0 18 0 9 0 869268331 428933120 20656 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554434 18446604469005535936 0 0 -1 5 0 0 0
[pid=12691/tid=12808] ppid=12689 vsize=418880 CPUtime=0 cores=1,3,5,7
/proc/12691/task/12808/stat : 12808 (python) S 12689 12691 11775 0 -1 4202560 28 929 0 0 0 0 0 0 20 0 9 0 869278023 428933120 20656 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554434 18446604469005535936 0 0 -1 7 0 0 0
[pid=12691/tid=12841] ppid=12689 vsize=418880 CPUtime=0 cores=1,3,5,7
/proc/12691/task/12841/stat : 12841 (python) S 12689 12691 11775 0 -1 4202560 26 929 0 0 0 0 0 0 20 0 9 0 869301055 428933120 20656 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554434 18446604469005535936 0 0 -1 1 0 0 0
[pid=12691/tid=12867] ppid=12689 vsize=418880 CPUtime=0 cores=1,3,5,7
/proc/12691/task/12867/stat : 12867 (python) S 12689 12691 11775 0 -1 4202560 28 929 0 0 0 0 0 0 20 0 9 0 869356283 428933120 20656 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554434 18446604469005535936 0 0 -1 7 0 0 0
[pid=12691/tid=12904] ppid=12689 vsize=418880 CPUtime=0 cores=1,3,5,7
/proc/12691/task/12904/stat : 12904 (python) S 12689 12691 11775 0 -1 4202560 28 929 0 0 0 0 0 0 20 0 9 0 869415796 428933120 20656 33554432000 4194304 5683644 140734567911392 18446744073709551615 248101521841 0 0 16781312 33554434 18446604469005535936 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1640.75
Current children cumulated vsize (KiB) 16050412

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 12691 and gives
#  childrusage.ru_utime.tv_sec=15
#  childrusage.ru_utime.tv_usec=772602
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=187971
# CPU time returned by wait4() is 15.9606
# while last known CPU time is 1640.75
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1634.39
CPU time (s): 1640.75
CPU user time (s): 1626.86
CPU system time (s): 13.89
CPU usage (%): 100.389
Max. virtual memory (cumulated for all children) (KiB): 16050412

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 15.7726
system time used= 0.187971
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 39340
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1032
involuntary context switches= 225

runsolver used 10.6294 second user time and 21.1778 second system time

The end

Launcher Data 

Begin job on node123 at 2011-06-09 01:42:05
IDJOB=3482429
IDBENCH=1475
IDSOLVER=1971
FILE ID=node123/3482429-1307576525
RUNJOBID= node123-1307576358-11794
PBS_JOBID= 13547676
Free space on /tmp= 73900 MiB

SOLVER NAME= borg pb-dec-11.04.03
BENCH NAME= PB06//final/normalized-PB06/SATUNSAT-SMALLINT/web/uclid_pb_benchmarks/normalized-cache-ibm-q-full.all.ucl.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.dec.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3482429-1307576525/watcher-3482429-1307576525 -o /tmp/evaluation-result-3482429-1307576525/solver-3482429-1307576525 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3482429-1307576525.opb -seed 923846817 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 14c93c93dc53353f81a889ada18ba601
RANDOM SEED=923846817

node123.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      30090640 kB
Buffers:        146176 kB
Cached:         454864 kB
SwapCached:      10384 kB
Active:        2439352 kB
Inactive:       136892 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      30090640 kB
SwapTotal:    67111528 kB
SwapFree:     45617076 kB
Dirty:          170204 kB
Writeback:           0 kB
AnonPages:     1972388 kB
Mapped:          29588 kB
Slab:           135760 kB
PageTables:      79424 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 133435784 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73788 MiB
End job on node123 at 2011-06-09 02:09:22