Trace number 3482425

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg pb-dec-11.04.03? (MO) 1399.81 1392.18

General information on the benchmark

Namenormalized-PB06/SATUNSAT-SMALLINT/web/uclid_pb_benchmarks/
normalized-ooo.ex.mem.LdValue.ucl.opb
MD5SUM70ea01a607f0d01f6199fb93098b84d8
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Has Objective FunctionNO
SatisfiableNO
(Un)Satisfiability was provedYES
Best value of the objective function
Optimality of the best value was proved
Number of variables77106
Total number of constraints218779
Number of constraints which are clauses205559
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints13220
Minimum length of a constraint1
Maximum length of a constraint15
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 129
Number of bits of the biggest number in a constraint 8
Biggest sum of numbers in a constraint 510
Number of bits of the biggest sum of numbers9
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.35/0.56	c INFO: loaded portfolio from HOME/etc/borg-mix+class.dec.pickle
0.35/0.56	c INFO: solving HOME/instance-3482425-1307576526.opb
1.64/1.84	c INFO: parsing took 1.26 s
2.93/4.11	c INFO: features took 2.26 CPU seconds
2.93/4.13	c INFO: running clasp-1.3.7@0 for 45 with 1796 remaining (b = 0.30)
61.92/62.01	c INFO: wrote minion input file to HOME/tmpwQrQsT.minion
61.92/62.01	c INFO: running minion-0.12@0 for 45 with 1749 remaining (b = 0.07)
109.00/108.87	c INFO: running scip-2.0.1-spx@0 for 45 with 1691 remaining (b = 0.03)
161.31/161.00	c INFO: running scip-2.0.1-clp@0 for 45 with 1643 remaining (b = 0.02)
214.63/214.09	c INFO: running clasp-1.3.7@46 for 45 with 1595 remaining (b = 0.02)
261.15/260.33	c INFO: running wbo1.4b-fixed@0 for 45 with 1549 remaining (b = 0.11)
308.03/307.09	c INFO: running clasp-1.3.7@92 for 136 with 1502 remaining (b = 0.03)
445.22/443.70	c INFO: running wbo1.4a@0 for 45 with 1366 remaining (b = 0.02)
496.30/494.45	c INFO: running scip-2.0.1-clp@48 for 45 with 1315 remaining (b = 0.01)
541.40/539.34	c INFO: running sat4j-pb-v20101225@0 for 45 with 1271 remaining (b = 0.01)
588.02/584.05	c INFO: running scip-2.0.1-spx@47 for 45 with 1225 remaining (b = 0.01)
633.08/628.83	c INFO: running scip-2.0.1-spx@92 for 410 with 1180 remaining (b = 0.06)
1049.43/1043.39	c INFO: running scip-2.0.1-spx@503 for 547 with 769 remaining (b = 0.07)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3482425-1307576526/watcher-3482425-1307576526 -o /tmp/evaluation-result-3482425-1307576526/solver-3482425-1307576526 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3482425-1307576526.opb -seed 954643882 -budget 1800 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.61 0.52 0.18 2/505 24358
/proc/meminfo: memFree=30436004/32951124 swapFree=51712240/67111528
[pid=24358] ppid=24356 vsize=16240 CPUtime=0.01 cores=1,3,5,7
/proc/24358/stat : 24358 (python) R 24356 24358 23366 0 -1 4202496 1062 0 0 0 1 0 0 0 17 0 1 0 869255341 16629760 934 33554432000 4194304 5683644 140736319267568 18446744073709551615 4378761 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/24358/statm: 4060 934 381 364 0 547 0

[startup+0.079164 s]
/proc/loadavg: 1.61 0.52 0.18 2/505 24358
/proc/meminfo: memFree=30436004/32951124 swapFree=51712240/67111528
[pid=24358] ppid=24356 vsize=50300 CPUtime=0.05 cores=1,3,5,7
/proc/24358/stat : 24358 (python) S 24356 24358 23366 0 -1 4202496 2009 0 0 0 4 1 0 0 17 0 1 0 869255341 51507200 1769 33554432000 4194304 5683644 140736319267568 18446744073709551615 216634513408 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 0
/proc/24358/statm: 12575 1769 480 364 0 1275 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 50300

[startup+0.101208 s]
/proc/loadavg: 1.61 0.52 0.18 2/505 24358
/proc/meminfo: memFree=30436004/32951124 swapFree=51712240/67111528
[pid=24358] ppid=24356 vsize=94064 CPUtime=0.07 cores=1,3,5,7
/proc/24358/stat : 24358 (python) S 24356 24358 23366 0 -1 4202496 2795 0 1 0 5 2 0 0 17 0 1 0 869255341 96321536 2468 33554432000 4194304 5683644 140736319267568 18446744073709551615 216634515200 0 0 16781312 2 18446744071562362840 0 0 17 7 0 0 0
/proc/24358/statm: 23516 2468 788 364 0 1627 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 94064

[startup+0.301644 s]
/proc/loadavg: 1.61 0.52 0.18 2/505 24358
/proc/meminfo: memFree=30436004/32951124 swapFree=51712240/67111528
[pid=24358] ppid=24356 vsize=135800 CPUtime=0.2 cores=1,3,5,7
/proc/24358/stat : 24358 (python) S 24356 24358 23366 0 -1 4202496 5643 932 1 0 13 7 0 0 17 0 1 0 869255341 139059200 3992 33554432000 4194304 5683644 140736319267568 18446744073709551615 216634513408 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 0
/proc/24358/statm: 33950 3992 1056 364 0 2957 0
Current children cumulated CPU time (s) 0.2
Current children cumulated vsize (KiB) 135800

[startup+0.701039 s]
/proc/loadavg: 1.61 0.52 0.18 2/505 24358
/proc/meminfo: memFree=30436004/32951124 swapFree=51712240/67111528
[pid=24358] ppid=24356 vsize=295540 CPUtime=0.54 cores=1,3,5,7
/proc/24358/stat : 24358 (python) R 24356 24358 23366 0 -1 4202496 12324 932 1 0 43 11 0 0 17 0 1 0 869255341 302632960 10187 33554432000 4194304 5683644 140736319267568 18446744073709551615 4445115 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/24358/statm: 73885 10187 1781 364 0 8778 0
Current children cumulated CPU time (s) 0.54
Current children cumulated vsize (KiB) 295540

[startup+1.50093 s]
/proc/loadavg: 1.61 0.52 0.18 3/501 24388
/proc/meminfo: memFree=30494044/32951124 swapFree=51712240/67111528
[pid=24358] ppid=24356 vsize=323792 CPUtime=1.33 cores=1,3,5,7
/proc/24358/stat : 24358 (python) R 24356 24358 23366 0 -1 4202496 19181 932 1 0 121 12 0 0 18 0 1 0 869255341 331563008 17044 33554432000 4194304 5683644 140736319267568 18446744073709551615 47347545755219 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/24358/statm: 80948 17046 1781 364 0 15841 0
Current children cumulated CPU time (s) 1.33
Current children cumulated vsize (KiB) 323792

[startup+3.10065 s]
/proc/loadavg: 1.56 0.53 0.19 2/506 24398
/proc/meminfo: memFree=30340424/32951124 swapFree=51712240/67111528
[pid=24358] ppid=24356 vsize=335236 CPUtime=2.93 cores=1,3,5,7
/proc/24358/stat : 24358 (python) R 24356 24358 23366 0 -1 4202496 31107 932 1 0 280 13 0 0 20 0 1 0 869255341 343281664 20157 33554432000 4194304 5683644 140736319267568 18446744073709551615 216634174891 0 0 16781312 33554434 0 0 0 17 5 0 0 0
/proc/24358/statm: 83809 20157 1816 364 0 18702 0
Current children cumulated CPU time (s) 2.93
Current children cumulated vsize (KiB) 335236

[startup+6.30109 s]
/proc/loadavg: 1.56 0.53 0.19 3/509 24402
/proc/meminfo: memFree=30220008/32951124 swapFree=51712240/67111528
[pid=24358] ppid=24356 vsize=348924 CPUtime=3.97 cores=1,3,5,7
/proc/24358/stat : 24358 (python) S 24356 24358 23366 0 -1 4202496 32251 932 1 0 383 14 0 0 25 0 2 0 869255341 357298176 21072 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646342091 0 0 16781312 33554434 18446744073709551615 0 0 17 7 0 0 0
/proc/24358/statm: 87231 21072 1872 364 0 22118 0
[pid=24400] ppid=24358 vsize=338680 CPUtime=0 cores=1,3,5,7
/proc/24400/stat : 24400 (python) S 24358 24358 23366 0 -1 4202560 728 0 0 0 0 0 0 0 18 0 1 0 869255754 346808320 19547 33554432000 4194304 5683644 140736319267568 18446744073709551615 216634535439 0 0 16781312 33554946 15548820551223968005 0 0 17 7 0 0 0
/proc/24400/statm: 84670 19547 351 364 0 19557 0
[pid=24402] ppid=24400 vsize=35256 CPUtime=2.15 cores=1,3,5,7
/proc/24402/stat : 24402 (clasp-1.3.7-x86) R 24400 24402 24402 0 -1 4202496 10732 0 0 0 214 1 0 0 25 0 1 0 869255755 36102144 8405 33554432000 134512640 136139019 4288208864 18446744073709551615 134883109 0 0 16781312 16386 0 0 0 17 5 0 0 0
/proc/24402/statm: 8814 8405 184 398 0 8414 0
[pid=24358/tid=24401] ppid=24356 vsize=348924 CPUtime=0 cores=1,3,5,7
/proc/24358/task/24401/stat : 24401 (python) S 24356 24358 23366 0 -1 4202560 29 932 0 0 0 0 0 0 25 0 2 0 869255755 357298176 21072 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554434 18446604467281377344 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 6.12
Current children cumulated vsize (KiB) 722860

[startup+12.701 s]
/proc/loadavg: 1.63 0.58 0.20 3/512 24405
/proc/meminfo: memFree=30165252/32951124 swapFree=51712240/67111528
[pid=24358] ppid=24356 vsize=348924 CPUtime=3.97 cores=1,3,5,7
/proc/24358/stat : 24358 (python) S 24356 24358 23366 0 -1 4202496 32251 932 1 0 383 14 0 0 25 0 2 0 869255341 357298176 21072 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646342091 0 0 16781312 33554434 18446744073709551615 0 0 17 7 0 0 0
/proc/24358/statm: 87231 21072 1872 364 0 22118 0
[pid=24400] ppid=24358 vsize=338680 CPUtime=0.03 cores=1,3,5,7
/proc/24400/stat : 24400 (python) S 24358 24358 23366 0 -1 4202560 886 0 0 0 1 2 0 0 18 0 1 0 869255754 346808320 19571 33554432000 4194304 5683644 140736319267568 18446744073709551615 216634535439 0 0 16781312 33554946 5623753486446373647 0 0 17 7 0 0 0
/proc/24400/statm: 84670 19571 375 364 0 19557 0
[pid=24402] ppid=24400 vsize=36324 CPUtime=8.55 cores=1,3,5,7
/proc/24402/stat : 24402 (clasp-1.3.7-x86) R 24400 24402 24402 0 -1 4202496 11005 0 0 0 854 1 0 0 25 0 1 0 869255755 37195776 8678 33554432000 134512640 136139019 4288208864 18446744073709551615 134832391 0 0 16781312 16386 0 0 0 17 5 0 0 0
/proc/24402/statm: 9081 8678 184 398 0 8681 0
[pid=24358/tid=24401] ppid=24356 vsize=348924 CPUtime=0 cores=1,3,5,7
/proc/24358/task/24401/stat : 24401 (python) S 24356 24358 23366 0 -1 4202560 29 932 0 0 0 0 0 0 25 0 2 0 869255755 357298176 21072 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554434 18446604467281377344 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 12.55
Current children cumulated vsize (KiB) 723928

[startup+25.5007 s]
/proc/loadavg: 1.69 0.62 0.22 3/512 24405
/proc/meminfo: memFree=30151052/32951124 swapFree=51712240/67111528
[pid=24358] ppid=24356 vsize=348924 CPUtime=3.97 cores=1,3,5,7
/proc/24358/stat : 24358 (python) S 24356 24358 23366 0 -1 4202496 32251 932 1 0 383 14 0 0 25 0 2 0 869255341 357298176 21072 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646342091 0 0 16781312 33554434 18446744073709551615 0 0 17 7 0 0 0
/proc/24358/statm: 87231 21072 1872 364 0 22118 0
[pid=24400] ppid=24358 vsize=338680 CPUtime=0.07 cores=1,3,5,7
/proc/24400/stat : 24400 (python) S 24358 24358 23366 0 -1 4202560 890 0 0 0 3 4 0 0 18 0 1 0 869255754 346808320 19571 33554432000 4194304 5683644 140736319267568 18446744073709551615 216634535439 0 0 16781312 33554946 5623311482772007695 0 0 17 1 0 0 0
/proc/24400/statm: 84670 19571 375 364 0 19557 0
[pid=24402] ppid=24400 vsize=47032 CPUtime=21.35 cores=1,3,5,7
/proc/24402/stat : 24402 (clasp-1.3.7-x86) R 24400 24402 24402 0 -1 4202496 13698 0 0 0 2133 2 0 0 25 0 1 0 869255755 48160768 11371 33554432000 134512640 136139019 4288208864 18446744073709551615 134883785 0 0 16781312 16386 0 0 0 17 5 0 0 0
/proc/24402/statm: 11758 11371 184 398 0 11358 0
[pid=24358/tid=24401] ppid=24356 vsize=348924 CPUtime=0 cores=1,3,5,7
/proc/24358/task/24401/stat : 24401 (python) S 24356 24358 23366 0 -1 4202560 29 932 0 0 0 0 0 0 25 0 2 0 869255755 357298176 21072 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554434 18446604467281377344 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 25.39
Current children cumulated vsize (KiB) 734636

[startup+51.1192 s]
/proc/loadavg: 1.79 0.73 0.27 3/513 24407
/proc/meminfo: memFree=30141364/32951124 swapFree=51712240/67111528
[pid=24358] ppid=24356 vsize=348928 CPUtime=4.73 cores=1,3,5,7
/proc/24358/stat : 24358 (python) R 24356 24358 23366 0 -1 4202496 32875 932 1 0 459 14 0 0 18 0 2 0 869255341 357302272 21074 33554432000 4194304 5683644 140736319267568 18446744073709551615 4625161 0 0 16781312 33554434 18446744073709551615 0 0 17 5 0 0 0
/proc/24358/statm: 87232 21074 1873 364 0 22119 0
[pid=24400] ppid=24358 vsize=348924 CPUtime=0.2 cores=1,3,5,7
/proc/24400/stat : 24400 (python) S 24358 24358 23366 0 -1 4202560 1129 0 0 0 9 11 0 0 18 0 2 0 869255754 357298176 19612 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646342091 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/24400/statm: 87231 19612 411 364 0 22118 0
[pid=24402] ppid=24400 vsize=48224 CPUtime=46.21 cores=1,3,5,7
/proc/24402/stat : 24402 (clasp-1.3.7-x86) T 24400 24402 24402 0 -1 4202496 13991 0 0 0 4619 2 0 0 25 0 1 0 869255755 49381376 11664 33554432000 134512640 136139019 4288208864 18446744073709551615 134828798 0 0 16781312 16386 18446744071562691360 0 0 17 5 0 0 0
/proc/24402/statm: 12056 11664 184 398 0 11656 0
[pid=24400/tid=24407] ppid=24358 vsize=348924 CPUtime=0 cores=1,3,5,7
/proc/24400/task/24407/stat : 24407 (python) S 24358 24358 23366 0 -1 4202560 40 0 0 0 0 0 0 0 19 0 2 0 869260377 357298176 19612 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554946 18446604457623691648 0 0 -1 5 0 0 0
[pid=24358/tid=24401] ppid=24356 vsize=348928 CPUtime=0 cores=1,3,5,7
/proc/24358/task/24401/stat : 24401 (python) S 24356 24358 23366 0 -1 4202560 29 932 0 0 0 0 0 0 25 0 2 0 869255755 357302272 21074 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554434 18446604467281377344 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 51.14
Current children cumulated vsize (KiB) 746076

[startup+102.301 s]
/proc/loadavg: 1.91 0.93 0.36 3/520 24415
/proc/meminfo: memFree=28947960/32951124 swapFree=51712240/67111528
[pid=24358] ppid=24356 vsize=359180 CPUtime=15.62 cores=1,3,5,7
/proc/24358/stat : 24358 (python) S 24356 24358 23366 0 -1 4202496 33193 932 1 0 1545 17 0 0 25 0 3 0 869255341 367800320 21079 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646342091 0 0 16781312 33554434 18446744073709551615 0 0 17 5 0 0 0
/proc/24358/statm: 89795 21079 1875 364 0 24679 0
[pid=24400] ppid=24358 vsize=348924 CPUtime=0.2 cores=1,3,5,7
/proc/24400/stat : 24400 (python) S 24358 24358 23366 0 -1 4202560 1129 0 0 0 9 11 0 0 18 0 2 0 869255754 357298176 19612 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646342091 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/24400/statm: 87231 19612 411 364 0 22118 0
[pid=24402] ppid=24400 vsize=48224 CPUtime=46.21 cores=1,3,5,7
/proc/24402/stat : 24402 (clasp-1.3.7-x86) T 24400 24402 24402 0 -1 4202496 13991 0 0 0 4619 2 0 0 25 0 1 0 869255755 49381376 11664 33554432000 134512640 136139019 4288208864 18446744073709551615 134828798 0 0 16781312 16386 18446744071562691360 0 0 17 5 0 0 0
/proc/24402/statm: 12056 11664 184 398 0 11656 0
[pid=24400/tid=24407] ppid=24358 vsize=348924 CPUtime=0 cores=1,3,5,7
/proc/24400/task/24407/stat : 24407 (python) S 24358 24358 23366 0 -1 4202560 40 0 0 0 0 0 0 0 19 0 2 0 869260377 357298176 19612 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554946 18446604457623691648 0 0 -1 5 0 0 0
[pid=24412] ppid=24358 vsize=348936 CPUtime=0.17 cores=1,3,5,7
/proc/24412/stat : 24412 (python) S 24358 24358 23366 0 -1 4202560 936 0 0 0 8 9 0 0 15 0 1 0 869261542 357310464 19572 33554432000 4194304 5683644 140736319267568 18446744073709551615 216634535494 0 0 16781312 33554946 5623753486446373647 0 0 17 7 0 0 0
/proc/24412/statm: 87234 19572 372 364 0 22118 0
[pid=24414] ppid=24412 vsize=275608 CPUtime=40.28 cores=1,3,5,7
/proc/24414/stat : 24414 (minion) R 24412 24414 24414 0 -1 4202496 77271 0 0 0 4017 11 0 0 25 0 1 0 869261543 282222592 61187 33554432000 134512640 165565024 4291176160 18446744073709551615 142764909 0 0 16781312 8396802 0 0 0 17 1 0 0 0
/proc/24414/statm: 68902 61187 346 7582 0 61317 0
[pid=24358/tid=24401] ppid=24356 vsize=359180 CPUtime=0 cores=1,3,5,7
/proc/24358/task/24401/stat : 24401 (python) S 24356 24358 23366 0 -1 4202560 29 932 0 0 0 0 0 0 25 0 3 0 869255755 367800320 21079 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554434 18446604467281377344 0 0 -1 7 0 0 0
[pid=24358/tid=24413] ppid=24356 vsize=359180 CPUtime=0 cores=1,3,5,7
/proc/24358/task/24413/stat : 24413 (python) S 24356 24358 23366 0 -1 4202560 26 932 0 0 0 0 0 0 25 0 3 0 869261543 367800320 21079 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554434 18446604467281377344 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 102.48
Current children cumulated vsize (KiB) 1380872

[startup+162.302 s]
/proc/loadavg: 2.06 1.16 0.48 3/532 24429
/proc/meminfo: memFree=27594008/32951124 swapFree=51712240/67111528
[pid=24358] ppid=24356 vsize=379692 CPUtime=15.68 cores=1,3,5,7
/proc/24358/stat : 24358 (python) S 24356 24358 23366 0 -1 4202496 35053 932 1 0 1550 18 0 0 20 0 5 0 869255341 388804608 21091 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646342091 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0
/proc/24358/statm: 94923 21091 1879 364 0 29801 0
[pid=24400] ppid=24358 vsize=348924 CPUtime=0.2 cores=1,3,5,7
/proc/24400/stat : 24400 (python) S 24358 24358 23366 0 -1 4202560 1129 0 0 0 9 11 0 0 18 0 2 0 869255754 357298176 19612 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646342091 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/24400/statm: 87231 19612 411 364 0 22118 0
[pid=24402] ppid=24400 vsize=48224 CPUtime=46.21 cores=1,3,5,7
/proc/24402/stat : 24402 (clasp-1.3.7-x86) T 24400 24402 24402 0 -1 4202496 13991 0 0 0 4619 2 0 0 25 0 1 0 869255755 49381376 11664 33554432000 134512640 136139019 4288208864 18446744073709551615 134828798 0 0 16781312 16386 18446744071562691360 0 0 17 5 0 0 0
/proc/24402/statm: 12056 11664 184 398 0 11656 0
[pid=24400/tid=24407] ppid=24358 vsize=348924 CPUtime=0 cores=1,3,5,7
/proc/24400/task/24407/stat : 24407 (python) S 24358 24358 23366 0 -1 4202560 40 0 0 0 0 0 0 0 19 0 2 0 869260377 357298176 19612 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554946 18446604457623691648 0 0 -1 5 0 0 0
[pid=24412] ppid=24358 vsize=348936 CPUtime=0.2 cores=1,3,5,7
/proc/24412/stat : 24412 (python) S 24358 24358 23366 0 -1 4202560 1107 0 0 0 10 10 0 0 17 0 2 0 869261542 357310464 19597 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646342091 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/24412/statm: 87234 19597 396 364 0 22118 0
[pid=24414] ppid=24412 vsize=275608 CPUtime=46.83 cores=1,3,5,7
/proc/24414/stat : 24414 (minion) T 24412 24414 24414 0 -1 4202496 77271 0 0 0 4672 11 0 0 25 0 1 0 869261543 282222592 61187 33554432000 134512640 165565024 4291176160 18446744073709551615 158624694 0 0 16781312 8396802 18446744071562691360 0 0 17 1 0 0 0
/proc/24414/statm: 68902 61187 346 7582 0 61317 0
[pid=24412/tid=24417] ppid=24358 vsize=348936 CPUtime=0 cores=1,3,5,7

################
# More data... #
################

[pid=24463/tid=24475] ppid=24461 vsize=8540476 CPUtime=0 cores=1,3,5,7
/proc/24463/task/24475/stat : 24475 (java) T 24461 24463 24463 0 -1 4202560 9 0 0 0 0 0 0 0 15 0 19 0 869309279 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646331289 0 4 0 16800975 18446744071562691360 0 0 -1 5 0 0 4
[pid=24463/tid=24476] ppid=24461 vsize=8540476 CPUtime=0 cores=1,3,5,7
/proc/24463/task/24476/stat : 24476 (java) T 24461 24463 24463 0 -1 4202560 1 0 0 0 0 0 0 0 18 0 19 0 869309281 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646338993 0 4 0 16800975 18446744071562691360 0 0 -1 5 0 0 0
[pid=24463/tid=24477] ppid=24461 vsize=8540476 CPUtime=0.78 cores=1,3,5,7
/proc/24463/task/24477/stat : 24477 (java) T 24461 24463 24463 0 -1 4202560 3241 0 0 0 78 0 0 0 15 0 19 0 869309281 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646331289 0 4 0 16800975 18446744071562691360 0 0 -1 3 0 0 0
[pid=24463/tid=24478] ppid=24461 vsize=8540476 CPUtime=0.5 cores=1,3,5,7
/proc/24463/task/24478/stat : 24478 (java) T 24461 24463 24463 0 -1 4202560 3633 0 0 0 50 0 0 0 15 0 19 0 869309281 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646331289 0 4 0 16800975 18446744071562691360 0 0 -1 1 0 0 0
[pid=24463/tid=24479] ppid=24461 vsize=8540476 CPUtime=0 cores=1,3,5,7
/proc/24463/task/24479/stat : 24479 (java) T 24461 24463 24463 0 -1 4202560 1 0 0 0 0 0 0 0 18 0 19 0 869309281 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646331289 0 4 0 16800975 18446744071562691360 0 0 -1 5 0 0 0
[pid=24463/tid=24480] ppid=24461 vsize=8540476 CPUtime=0 cores=1,3,5,7
/proc/24463/task/24480/stat : 24480 (java) T 24461 24463 24463 0 -1 4202560 7 0 0 0 0 0 0 0 15 0 19 0 869309281 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646331904 0 4 0 16800975 18446744071562691360 0 0 -1 3 0 0 0
[pid=24463/tid=24481] ppid=24461 vsize=8540476 CPUtime=0 cores=1,3,5,7
/proc/24463/task/24481/stat : 24481 (java) T 24461 24463 24463 0 -1 4202560 7 0 0 0 0 0 0 0 18 0 19 0 869309428 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646331904 0 4 0 16800975 18446744071562691360 0 0 -1 7 0 0 0
[pid=24461/tid=24484] ppid=24358 vsize=400216 CPUtime=0 cores=1,3,5,7
/proc/24461/task/24484/stat : 24484 (python) S 24358 24358 23366 0 -1 4202560 45 0 0 0 0 0 0 0 17 0 2 0 869313745 409821184 19617 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554946 18446604460774859968 0 0 -1 7 0 0 0
[pid=24358/tid=24401] ppid=24356 vsize=410460 CPUtime=0 cores=1,3,5,7
/proc/24358/task/24401/stat : 24401 (python) S 24356 24358 23366 0 -1 4202560 37 932 0 0 0 0 0 0 18 0 8 0 869255755 420311040 21109 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554434 18446604467281377344 0 0 -1 7 0 0 0
[pid=24358/tid=24413] ppid=24356 vsize=410460 CPUtime=0 cores=1,3,5,7
/proc/24358/task/24413/stat : 24413 (python) S 24356 24358 23366 0 -1 4202560 26 932 0 0 0 0 0 0 25 0 8 0 869261543 420311040 21109 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554434 18446604467281377344 0 0 -1 5 0 0 0
[pid=24358/tid=24419] ppid=24356 vsize=410460 CPUtime=0 cores=1,3,5,7
/proc/24358/task/24419/stat : 24419 (python) S 24356 24358 23366 0 -1 4202560 33 932 0 0 0 0 0 0 18 0 8 0 869266229 420311040 21109 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554434 18446604467281377344 0 0 -1 1 0 0 0
[pid=24358/tid=24428] ppid=24356 vsize=410460 CPUtime=0 cores=1,3,5,7
/proc/24358/task/24428/stat : 24428 (python) S 24356 24358 23366 0 -1 4202560 36 932 0 0 0 0 0 0 18 0 8 0 869271442 420311040 21109 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554434 18446604467281377344 0 0 -1 1 0 0 0
[pid=24358/tid=24439] ppid=24356 vsize=410460 CPUtime=0 cores=1,3,5,7
/proc/24358/task/24439/stat : 24439 (python) S 24356 24358 23366 0 -1 4202560 29 932 0 0 0 0 0 0 20 0 8 0 869281374 420311040 21109 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554434 18446604467281377344 0 0 -1 1 0 0 0
[pid=24358/tid=24452] ppid=24356 vsize=410460 CPUtime=0 cores=1,3,5,7
/proc/24358/task/24452/stat : 24452 (python) S 24356 24358 23366 0 -1 4202560 26 932 0 0 0 0 0 0 20 0 8 0 869299712 420311040 21109 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554434 18446604467281377344 0 0 -1 1 0 0 0
[pid=24358/tid=24462] ppid=24356 vsize=410460 CPUtime=0 cores=1,3,5,7
/proc/24358/task/24462/stat : 24462 (python) S 24356 24358 23366 0 -1 4202560 26 932 0 0 0 0 0 0 20 0 8 0 869309275 420311040 21109 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554434 18446604467281377344 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1369.88
Current children cumulated vsize (KiB) 15709220



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1392.1 s]
/proc/loadavg: 2.09 2.05 1.62 3/580 24522
/proc/meminfo: memFree=19797356/32951124 swapFree=51712240/67111528
[pid=24358] ppid=24356 vsize=410460 CPUtime=15.87 cores=1,3,5,7
/proc/24358/stat : 24358 (python) S 24356 24358 23366 0 -1 4202496 38674 932 1 0 1568 19 0 0 18 0 8 0 869255341 420311040 21109 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646342091 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0
/proc/24358/statm: 102615 21109 1885 364 0 37484 0
[pid=24400] ppid=24358 vsize=348924 CPUtime=1 cores=1,3,5,7
/proc/24400/stat : 24400 (python) S 24358 24358 23366 0 -1 4202560 1204 0 0 0 46 54 0 0 18 0 2 0 869255754 357298176 19613 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646342091 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/24400/statm: 87231 19613 412 364 0 22118 0
[pid=24402] ppid=24400 vsize=92944 CPUtime=229.02 cores=1,3,5,7
/proc/24402/stat : 24402 (clasp-1.3.7-x86) T 24400 24402 24402 0 -1 4202496 25069 0 0 0 22897 5 0 0 25 0 1 0 869255755 95174656 22742 33554432000 134512640 136139019 4288208864 18446744073709551615 134608226 0 0 16781312 16386 18446744071562691360 0 0 17 5 0 0 0
/proc/24402/statm: 23236 22742 184 398 0 22836 0
[pid=24400/tid=24407] ppid=24358 vsize=348924 CPUtime=0 cores=1,3,5,7
/proc/24400/task/24407/stat : 24407 (python) S 24358 24358 23366 0 -1 4202560 42 0 0 0 0 0 0 0 18 0 2 0 869260377 357298176 19613 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554946 18446604457623691648 0 0 -1 5 0 0 0
[pid=24412] ppid=24358 vsize=348936 CPUtime=0.2 cores=1,3,5,7
/proc/24412/stat : 24412 (python) S 24358 24358 23366 0 -1 4202560 1107 0 0 0 10 10 0 0 17 0 2 0 869261542 357310464 19597 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646342091 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/24412/statm: 87234 19597 396 364 0 22118 0
[pid=24414] ppid=24412 vsize=275608 CPUtime=46.83 cores=1,3,5,7
/proc/24414/stat : 24414 (minion) T 24412 24414 24414 0 -1 4202496 77271 0 0 0 4672 11 0 0 25 0 1 0 869261543 282222592 61187 33554432000 134512640 165565024 4291176160 18446744073709551615 158624694 0 0 16781312 8396802 18446744071562691360 0 0 17 1 0 0 0
/proc/24414/statm: 68902 61187 346 7582 0 61317 0
[pid=24412/tid=24417] ppid=24358 vsize=348936 CPUtime=0 cores=1,3,5,7
/proc/24412/task/24417/stat : 24417 (python) S 24358 24358 23366 0 -1 4202560 50 0 0 0 0 0 0 0 17 0 2 0 869266226 357310464 19597 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554946 18446604463087251648 0 0 -1 7 0 0 0
[pid=24418] ppid=24358 vsize=359192 CPUtime=3.86 cores=1,3,5,7
/proc/24418/stat : 24418 (python) S 24358 24358 23366 0 -1 4202560 1189 0 0 0 188 198 0 0 15 0 2 0 869266228 367812608 19601 33554432000 4194304 5683644 140736319267568 18446744073709551615 216634535494 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/24418/statm: 89798 19601 396 364 0 24679 0
[pid=24420] ppid=24418 vsize=2744744 CPUtime=860.14 cores=1,3,5,7
/proc/24420/stat : 24420 (scip-2.0.1.linu) R 24418 24420 24420 0 -1 4202496 683161 0 0 0 85177 837 0 0 25 0 1 0 869266229 2810617856 675243 33554432000 4194304 13684479 140733414326544 18446744073709551615 11876362 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/24420/statm: 686186 675243 659 2317 0 683864 0
[pid=24418/tid=24426] ppid=24358 vsize=359192 CPUtime=0 cores=1,3,5,7
/proc/24418/task/24426/stat : 24426 (python) S 24358 24358 23366 0 -1 4202560 44 0 0 0 0 0 0 0 15 0 2 0 869271439 367812608 19601 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554946 18446604469640069504 0 0 -1 1 0 0 0
[pid=24427] ppid=24358 vsize=369448 CPUtime=0.45 cores=1,3,5,7
/proc/24427/stat : 24427 (python) S 24358 24358 23366 0 -1 4202560 1170 0 0 0 22 23 0 0 15 0 2 0 869271442 378314752 19605 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646342091 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/24427/statm: 92362 19605 396 364 0 27240 0
[pid=24429] ppid=24427 vsize=994744 CPUtime=97.92 cores=1,3,5,7
/proc/24429/stat : 24429 (scip-2.0.1.linu) T 24427 24429 24429 0 -1 4202496 248926 0 0 0 9327 465 0 0 25 0 1 0 869271442 1018617856 241011 33554432000 4194304 15243407 140736659643280 18446744073709551615 4880915 0 0 16781312 2 18446744071562691360 0 0 17 3 0 0 0
/proc/24429/statm: 248686 241011 702 2698 0 245981 0
[pid=24427/tid=24435] ppid=24358 vsize=369448 CPUtime=0 cores=1,3,5,7
/proc/24427/task/24435/stat : 24435 (python) S 24358 24358 23366 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 869276748 378314752 19605 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554946 18446604467019464896 0 0 -1 3 0 0 0
[pid=24438] ppid=24358 vsize=379704 CPUtime=0.2 cores=1,3,5,7
/proc/24438/stat : 24438 (python) S 24358 24358 23366 0 -1 4202560 1137 0 0 0 9 11 0 0 16 0 2 0 869281374 388816896 19609 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646342091 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/24438/statm: 94926 19609 396 364 0 29801 0
[pid=24440] ppid=24438 vsize=140500 CPUtime=46.72 cores=1,3,5,7
/proc/24440/stat : 24440 (wbo1.4b-fixed) T 24438 24440 24440 0 -1 4202496 36582 0 0 0 4635 37 0 0 25 0 1 0 869281375 143872000 33961 33554432000 134512640 135752315 4292951952 18446744073709551615 134517428 0 0 16781312 3 18446744071562691360 0 0 17 3 0 0 0
/proc/24440/statm: 35125 33961 146 303 0 34820 0
[pid=24438/tid=24446] ppid=24358 vsize=379704 CPUtime=0 cores=1,3,5,7
/proc/24438/task/24446/stat : 24446 (python) S 24358 24358 23366 0 -1 4202560 45 0 0 0 0 0 0 0 16 0 2 0 869286048 388816896 19609 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554946 18446604462109786240 0 0 -1 7 0 0 0
[pid=24451] ppid=24358 vsize=389960 CPUtime=0.23 cores=1,3,5,7
/proc/24451/stat : 24451 (python) S 24358 24358 23366 0 -1 4202560 1140 0 0 0 12 11 0 0 17 0 2 0 869299711 399319040 19613 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646342091 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/24451/statm: 97490 19613 396 364 0 32362 0
[pid=24453] ppid=24451 vsize=76776 CPUtime=50.71 cores=1,3,5,7
/proc/24453/stat : 24453 (wbo1.4a) T 24451 24453 24453 0 -1 4202496 20693 0 0 0 5040 31 0 0 25 0 1 0 869299712 78618624 18074 33554432000 134512640 135751879 4288011168 18446744073709551615 134604644 0 0 16781312 3 18446744071562691360 0 0 17 3 0 0 0
/proc/24453/statm: 19194 18074 143 303 0 18889 0
[pid=24451/tid=24459] ppid=24358 vsize=389960 CPUtime=0 cores=1,3,5,7
/proc/24451/task/24459/stat : 24459 (python) S 24358 24358 23366 0 -1 4202560 46 0 0 0 0 0 0 0 17 0 2 0 869304784 399319040 19613 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554946 18446604469923733760 0 0 -1 3 0 0 0
[pid=24461] ppid=24358 vsize=400216 CPUtime=0.21 cores=1,3,5,7
/proc/24461/stat : 24461 (python) S 24358 24358 23366 0 -1 4202560 1143 0 0 0 10 11 0 0 17 0 2 0 869309275 409821184 19617 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646342091 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/24461/statm: 100054 19617 396 364 0 34923 0
[pid=24463] ppid=24461 vsize=8540476 CPUtime=46.45 cores=1,3,5,7
/proc/24463/stat : 24463 (java) T 24461 24463 24463 0 -1 4202496 505195 0 1 0 4591 54 0 0 15 0 19 0 869309276 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646318053 0 0 0 16800975 18446744073709551615 0 0 17 5 0 0 0
/proc/24463/statm: 2135119 501209 2255 9 0 2125860 0
[pid=24463/tid=24464] ppid=24461 vsize=8540476 CPUtime=43.43 cores=1,3,5,7
/proc/24463/task/24464/stat : 24464 (java) T 24461 24463 24463 0 -1 4202560 446076 0 1 0 4296 47 0 0 25 0 19 0 869309277 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 46912505881138 0 4 0 16800975 18446744071562691360 0 0 -1 1 0 0 0
[pid=24463/tid=24465] ppid=24461 vsize=8540476 CPUtime=0.06 cores=1,3,5,7
/proc/24463/task/24465/stat : 24465 (java) T 24461 24463 24463 0 -1 4202560 3294 0 0 0 6 0 0 0 15 0 19 0 869309277 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646331289 0 4 0 16800975 18446744071562691360 0 0 -1 3 0 0 0
[pid=24463/tid=24466] ppid=24461 vsize=8540476 CPUtime=0.08 cores=1,3,5,7
/proc/24463/task/24466/stat : 24466 (java) T 24461 24463 24463 0 -1 4202560 5001 0 0 0 8 0 0 0 15 0 19 0 869309277 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646331289 0 4 0 16800975 18446744071562691360 0 0 -1 7 0 0 0
[pid=24463/tid=24467] ppid=24461 vsize=8540476 CPUtime=0.1 cores=1,3,5,7
/proc/24463/task/24467/stat : 24467 (java) T 24461 24463 24463 0 -1 4202560 4249 0 0 0 10 0 0 0 15 0 19 0 869309277 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646331289 0 4 0 16800975 18446744071562691360 0 0 -1 3 0 0 0
[pid=24463/tid=24468] ppid=24461 vsize=8540476 CPUtime=0.06 cores=1,3,5,7
/proc/24463/task/24468/stat : 24468 (java) T 24461 24463 24463 0 -1 4202560 3001 0 0 0 6 0 0 0 15 0 19 0 869309277 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646331289 0 4 0 16800975 18446744071562691360 0 0 -1 7 0 0 0
[pid=24463/tid=24469] ppid=24461 vsize=8540476 CPUtime=0.11 cores=1,3,5,7
/proc/24463/task/24469/stat : 24469 (java) T 24461 24463 24463 0 -1 4202560 5733 0 0 0 11 0 0 0 15 0 19 0 869309277 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646331289 0 4 0 16800975 18446744071562691360 0 0 -1 3 0 0 0
[pid=24463/tid=24470] ppid=24461 vsize=8540476 CPUtime=0.05 cores=1,3,5,7
/proc/24463/task/24470/stat : 24470 (java) T 24461 24463 24463 0 -1 4202560 2360 0 0 0 5 0 0 0 15 0 19 0 869309277 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646331289 0 4 0 16800975 18446744071562691360 0 0 -1 5 0 0 0
[pid=24463/tid=24471] ppid=24461 vsize=8540476 CPUtime=0.09 cores=1,3,5,7
/proc/24463/task/24471/stat : 24471 (java) T 24461 24463 24463 0 -1 4202560 5928 0 0 0 9 0 0 0 15 0 19 0 869309277 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646331289 0 4 0 16800975 18446744071562691360 0 0 -1 5 0 0 0
[pid=24463/tid=24472] ppid=24461 vsize=8540476 CPUtime=0.09 cores=1,3,5,7
/proc/24463/task/24472/stat : 24472 (java) T 24461 24463 24463 0 -1 4202560 6142 0 0 0 9 0 0 0 15 0 19 0 869309277 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646331289 0 4 0 16800975 18446744071562691360 0 0 -1 5 0 0 0
[pid=24463/tid=24473] ppid=24461 vsize=8540476 CPUtime=1 cores=1,3,5,7
/proc/24463/task/24473/stat : 24473 (java) T 24461 24463 24463 0 -1 4202560 15141 0 0 0 98 2 0 0 15 0 19 0 869309278 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646331904 0 0 0 16800975 18446744071562691360 0 0 -1 1 0 0 0
[pid=24463/tid=24474] ppid=24461 vsize=8540476 CPUtime=0 cores=1,3,5,7
/proc/24463/task/24474/stat : 24474 (java) T 24461 24463 24463 0 -1 4202560 17 0 0 0 0 0 0 0 15 0 19 0 869309279 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646331289 0 4 0 16800975 18446744071562691360 0 0 -1 5 0 0 0
[pid=24463/tid=24475] ppid=24461 vsize=8540476 CPUtime=0 cores=1,3,5,7
/proc/24463/task/24475/stat : 24475 (java) T 24461 24463 24463 0 -1 4202560 9 0 0 0 0 0 0 0 15 0 19 0 869309279 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646331289 0 4 0 16800975 18446744071562691360 0 0 -1 5 0 0 4
[pid=24463/tid=24476] ppid=24461 vsize=8540476 CPUtime=0 cores=1,3,5,7
/proc/24463/task/24476/stat : 24476 (java) T 24461 24463 24463 0 -1 4202560 1 0 0 0 0 0 0 0 18 0 19 0 869309281 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646338993 0 4 0 16800975 18446744071562691360 0 0 -1 5 0 0 0
[pid=24463/tid=24477] ppid=24461 vsize=8540476 CPUtime=0.78 cores=1,3,5,7
/proc/24463/task/24477/stat : 24477 (java) T 24461 24463 24463 0 -1 4202560 3241 0 0 0 78 0 0 0 15 0 19 0 869309281 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646331289 0 4 0 16800975 18446744071562691360 0 0 -1 3 0 0 0
[pid=24463/tid=24478] ppid=24461 vsize=8540476 CPUtime=0.5 cores=1,3,5,7
/proc/24463/task/24478/stat : 24478 (java) T 24461 24463 24463 0 -1 4202560 3633 0 0 0 50 0 0 0 15 0 19 0 869309281 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646331289 0 4 0 16800975 18446744071562691360 0 0 -1 1 0 0 0
[pid=24463/tid=24479] ppid=24461 vsize=8540476 CPUtime=0 cores=1,3,5,7
/proc/24463/task/24479/stat : 24479 (java) T 24461 24463 24463 0 -1 4202560 1 0 0 0 0 0 0 0 18 0 19 0 869309281 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646331289 0 4 0 16800975 18446744071562691360 0 0 -1 5 0 0 0
[pid=24463/tid=24480] ppid=24461 vsize=8540476 CPUtime=0 cores=1,3,5,7
/proc/24463/task/24480/stat : 24480 (java) T 24461 24463 24463 0 -1 4202560 7 0 0 0 0 0 0 0 15 0 19 0 869309281 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646331904 0 4 0 16800975 18446744071562691360 0 0 -1 3 0 0 0
[pid=24463/tid=24481] ppid=24461 vsize=8540476 CPUtime=0 cores=1,3,5,7
/proc/24463/task/24481/stat : 24481 (java) T 24461 24463 24463 0 -1 4202560 7 0 0 0 0 0 0 0 18 0 19 0 869309428 8745447424 501209 33554432000 1073741824 1073778376 140735756588352 18446744073709551615 216646331904 0 4 0 16800975 18446744071562691360 0 0 -1 7 0 0 0
[pid=24461/tid=24484] ppid=24358 vsize=400216 CPUtime=0 cores=1,3,5,7
/proc/24461/task/24484/stat : 24484 (python) S 24358 24358 23366 0 -1 4202560 45 0 0 0 0 0 0 0 17 0 2 0 869313745 409821184 19617 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554946 18446604460774859968 0 0 -1 7 0 0 0
[pid=24358/tid=24401] ppid=24356 vsize=410460 CPUtime=0 cores=1,3,5,7
/proc/24358/task/24401/stat : 24401 (python) S 24356 24358 23366 0 -1 4202560 37 932 0 0 0 0 0 0 18 0 8 0 869255755 420311040 21109 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554434 18446604467281377344 0 0 -1 7 0 0 0
[pid=24358/tid=24413] ppid=24356 vsize=410460 CPUtime=0 cores=1,3,5,7
/proc/24358/task/24413/stat : 24413 (python) S 24356 24358 23366 0 -1 4202560 26 932 0 0 0 0 0 0 25 0 8 0 869261543 420311040 21109 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554434 18446604467281377344 0 0 -1 5 0 0 0
[pid=24358/tid=24419] ppid=24356 vsize=410460 CPUtime=0 cores=1,3,5,7
/proc/24358/task/24419/stat : 24419 (python) S 24356 24358 23366 0 -1 4202560 33 932 0 0 0 0 0 0 18 0 8 0 869266229 420311040 21109 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554434 18446604467281377344 0 0 -1 1 0 0 0
[pid=24358/tid=24428] ppid=24356 vsize=410460 CPUtime=0 cores=1,3,5,7
/proc/24358/task/24428/stat : 24428 (python) S 24356 24358 23366 0 -1 4202560 36 932 0 0 0 0 0 0 18 0 8 0 869271442 420311040 21109 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554434 18446604467281377344 0 0 -1 1 0 0 0
[pid=24358/tid=24439] ppid=24356 vsize=410460 CPUtime=0 cores=1,3,5,7
/proc/24358/task/24439/stat : 24439 (python) S 24356 24358 23366 0 -1 4202560 29 932 0 0 0 0 0 0 20 0 8 0 869281374 420311040 21109 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554434 18446604467281377344 0 0 -1 1 0 0 0
[pid=24358/tid=24452] ppid=24356 vsize=410460 CPUtime=0 cores=1,3,5,7
/proc/24358/task/24452/stat : 24452 (python) S 24356 24358 23366 0 -1 4202560 26 932 0 0 0 0 0 0 20 0 8 0 869299712 420311040 21109 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554434 18446604467281377344 0 0 -1 1 0 0 0
[pid=24358/tid=24462] ppid=24356 vsize=410460 CPUtime=0 cores=1,3,5,7
/proc/24358/task/24462/stat : 24462 (python) S 24356 24358 23366 0 -1 4202560 26 932 0 0 0 0 0 0 20 0 8 0 869309275 420311040 21109 33554432000 4194304 5683644 140736319267568 18446744073709551615 216646338993 0 0 16781312 33554434 18446604467281377344 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1399.81
Current children cumulated vsize (KiB) 15872632

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 24358 and gives
#  childrusage.ru_utime.tv_sec=15
#  childrusage.ru_utime.tv_usec=683615
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=203968
# CPU time returned by wait4() is 15.8876
# while last known CPU time is 1399.81
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1392.18
CPU time (s): 1399.81
CPU user time (s): 1382.04
CPU system time (s): 17.77
CPU usage (%): 100.548
Max. virtual memory (cumulated for all children) (KiB): 15872632

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 15.6836
system time used= 0.203968
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 39610
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3788
involuntary context switches= 37

runsolver used 9.77651 second user time and 23.1355 second system time

The end

Launcher Data 

Begin job on node103 at 2011-06-09 01:42:07
IDJOB=3482425
IDBENCH=1481
IDSOLVER=1971
FILE ID=node103/3482425-1307576526
RUNJOBID= node103-1307576441-23385
PBS_JOBID= 13547702
Free space on /tmp= 73948 MiB

SOLVER NAME= borg pb-dec-11.04.03
BENCH NAME= PB06//final/normalized-PB06/SATUNSAT-SMALLINT/web/uclid_pb_benchmarks/normalized-ooo.ex.mem.LdValue.ucl.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.dec.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3482425-1307576526/watcher-3482425-1307576526 -o /tmp/evaluation-result-3482425-1307576526/solver-3482425-1307576526 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3482425-1307576526.opb -seed 954643882 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 70ea01a607f0d01f6199fb93098b84d8
RANDOM SEED=954643882

node103.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30436168 kB
Buffers:        175268 kB
Cached:         495968 kB
SwapCached:      23516 kB
Active:        2024308 kB
Inactive:       188428 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30436168 kB
SwapTotal:    67111528 kB
SwapFree:     51712240 kB
Dirty:          214280 kB
Writeback:           8 kB
AnonPages:     1539532 kB
Mapped:          25528 kB
Slab:           158676 kB
PageTables:      73660 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 162069380 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73716 MiB
End job on node103 at 2011-06-09 02:05:21