Trace number 3740074

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB11: borg pb-dec-11.04.03? (MO) 526.59 523.071

General information on the benchmark

Name/DEC-SMALLINT-LIN/oliveras/j60/
normalized-j6029_8-unsat.opb
MD5SUMc619d5d1f14c9c14bda19749a65da87b
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmarkUNSAT
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark1646.31
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables11712
Total number of constraints38651
Number of constraints which are clauses38271
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints380
Minimum length of a constraint1
Maximum length of a constraint60
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 24
Number of bits of the biggest number in a constraint 5
Biggest sum of numbers in a constraint 368
Number of bits of the biggest sum of numbers9
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.36/0.54	c INFO: loaded portfolio from HOME/etc/borg-mix+class.dec.pickle
0.36/0.54	c INFO: solving HOME/instance-3740074-1338719522.opb
0.56/0.76	c INFO: parsing took 0.21 s
0.96/1.16	c INFO: features took 0.40 CPU seconds
0.96/1.19	c INFO: running clasp-1.3.7@0 for 45 with 1799 remaining (b = 0.90)
50.19/50.31	c INFO: wrote minion input file to HOME/tmpYhypZU.minion
50.19/50.31	c INFO: running minion-0.12@0 for 45 with 1751 remaining (b = 0.35)
97.94/98.07	c INFO: running wbo1.4b-fixed@0 for 45 with 1702 remaining (b = 0.07)
147.19/147.22	c INFO: running scip-2.0.1-spx@0 for 45 with 1653 remaining (b = 0.03)
196.14/196.19	c INFO: running clasp-1.3.7@47 for 45 with 1604 remaining (b = 0.03)
241.39/241.33	c INFO: running wbo1.4a@0 for 45 with 1559 remaining (b = 0.05)
290.24/290.13	c INFO: running sat4j-pb-v20101225@0 for 45 with 1511 remaining (b = 0.02)
339.51/337.55	c INFO: running scip-2.0.1-clp@0 for 45 with 1461 remaining (b = 0.02)
386.57/384.56	c INFO: running clasp-1.3.7@92 for 136 with 1415 remaining (b = 0.05)
524.14/521.96	c INFO: running sat4j-pb-v20101225-cutting@0 for 45 with 1278 remaining (b = 0.02)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3740074-1338719522/watcher-3740074-1338719522 -o /tmp/evaluation-result-3740074-1338719522/solver-3740074-1338719522 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3740074-1338719522.opb -seed 1611718967 -budget 1800 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.97 2.01 2.00 3/200 580
/proc/meminfo: memFree=25586092/32873648 swapFree=7120/7120
[pid=580] ppid=578 vsize=15276 CPUtime=0 cores=0,2,4,6
/proc/580/stat : 580 (python) D 578 580 27000 0 -1 4202496 632 0 0 0 0 0 0 0 20 0 1 0 338079636 15642624 526 33554432000 4194304 5683644 140735443741184 140735443727096 219513963408 0 0 16781312 2 18446744072100609316 0 0 17 0 0 0 0 0 0
/proc/580/statm: 3819 526 312 364 0 261 0

[startup+0.073518 s]
/proc/loadavg: 1.97 2.01 2.00 3/200 580
/proc/meminfo: memFree=25586092/32873648 swapFree=7120/7120
[pid=580] ppid=578 vsize=45912 CPUtime=0.05 cores=0,2,4,6
/proc/580/stat : 580 (python) R 578 580 27000 0 -1 4202496 2066 0 0 0 4 1 0 0 20 0 1 0 338079636 47013888 1686 33554432000 4194304 5683644 140735443741184 140735443685416 219513631947 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/580/statm: 11478 1694 479 364 0 1182 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 45912

[startup+0.100268 s]
/proc/loadavg: 1.97 2.01 2.00 3/200 580
/proc/meminfo: memFree=25586092/32873648 swapFree=7120/7120
[pid=580] ppid=578 vsize=86164 CPUtime=0.06 cores=0,2,4,6
/proc/580/stat : 580 (python) R 578 580 27000 0 -1 4202496 2973 0 1 0 5 1 0 0 20 0 1 0 338079636 88231936 2476 33554432000 4194304 5683644 140735443741184 140735443667496 4940551 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/580/statm: 21541 2476 773 364 0 1615 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 86164

[startup+0.300197 s]
/proc/loadavg: 1.97 2.01 2.00 3/200 580
/proc/meminfo: memFree=25586092/32873648 swapFree=7120/7120
[pid=580] ppid=578 vsize=155644 CPUtime=0.21 cores=0,2,4,6
/proc/580/stat : 580 (python) R 578 580 27000 0 -1 4202496 6324 1013 1 0 16 5 0 0 20 0 1 0 338079636 159379456 4149 33554432000 4194304 5683644 140735443741184 140735443618312 219504752735 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/580/statm: 38911 4149 1144 364 0 3222 0
Current children cumulated CPU time (s) 0.21
Current children cumulated vsize (KiB) 155644

[startup+0.700206 s]
/proc/loadavg: 1.97 2.01 2.00 3/200 580
/proc/meminfo: memFree=25586092/32873648 swapFree=7120/7120
[pid=580] ppid=578 vsize=287668 CPUtime=0.56 cores=0,2,4,6
/proc/580/stat : 580 (python) R 578 580 27000 0 -1 4202496 11570 1013 1 0 47 9 0 0 20 0 1 0 338079636 294572032 8604 33554432000 4194304 5683644 140735443741184 140735443663256 4539731 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/580/statm: 71917 8604 1811 364 0 7146 0
Current children cumulated CPU time (s) 0.56
Current children cumulated vsize (KiB) 287668

[startup+1.50018 s]
/proc/loadavg: 1.97 2.01 2.00 3/201 585
/proc/meminfo: memFree=25555952/32873648 swapFree=7120/7120
[pid=580] ppid=578 vsize=366208 CPUtime=1.05 cores=0,2,4,6
/proc/580/stat : 580 (python) S 578 580 27000 0 -1 4202496 13949 1013 1 0 96 9 0 0 20 0 2 0 338079636 374996992 9414 33554432000 4194304 5683644 140735443741184 140735443736016 219517347085 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/580/statm: 91552 9414 1901 364 0 26775 0
Current children cumulated CPU time (s) 1.05
Current children cumulated vsize (KiB) 366208

[startup+3.10031 s]
/proc/loadavg: 1.97 2.01 2.00 3/204 588
/proc/meminfo: memFree=25549728/32873648 swapFree=7120/7120
[pid=580] ppid=578 vsize=366208 CPUtime=1.05 cores=0,2,4,6
/proc/580/stat : 580 (python) S 578 580 27000 0 -1 4202496 13949 1013 1 0 96 9 0 0 20 0 2 0 338079636 374996992 9414 33554432000 4194304 5683644 140735443741184 140735443736016 219517347085 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/580/statm: 91552 9414 1901 364 0 26775 0
[pid=586] ppid=580 vsize=290428 CPUtime=0 cores=0,2,4,6
/proc/586/stat : 586 (python) S 580 580 27000 0 -1 4202560 741 0 0 0 0 0 0 0 20 0 1 0 338079756 297398272 7847 33554432000 4194304 5683644 140735443741184 140735443732272 219513979304 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/586/statm: 72607 7847 340 364 0 7830 0
[pid=588] ppid=586 vsize=5600 CPUtime=1.89 cores=0,2,4,6
/proc/588/stat : 588 (clasp-1.3.7-x86) R 586 588 588 0 -1 4202496 1616 0 0 0 189 0 0 0 20 0 1 0 338079756 5734400 1154 33554432000 134512640 136139019 4287684720 4287683552 134832223 0 0 16781312 16386 0 0 0 17 0 0 0 0 0 0
/proc/588/statm: 1400 1154 183 398 0 1000 0
[pid=580/tid=587] ppid=578 vsize=366208 CPUtime=0 cores=0,2,4,6
/proc/580/task/587/stat : 587 (python) S 578 580 27000 0 -1 4202560 25 1013 0 0 0 0 0 0 20 0 2 0 338079756 374996992 9414 33554432000 4194304 5683644 140735443741184 140648976749104 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 2.94
Current children cumulated vsize (KiB) 662236

[startup+6.30031 s]
/proc/loadavg: 2.05 2.02 2.00 3/204 588
/proc/meminfo: memFree=25549852/32873648 swapFree=7120/7120
[pid=580] ppid=578 vsize=366208 CPUtime=1.05 cores=0,2,4,6
/proc/580/stat : 580 (python) S 578 580 27000 0 -1 4202496 13949 1013 1 0 96 9 0 0 20 0 2 0 338079636 374996992 9414 33554432000 4194304 5683644 140735443741184 140735443736016 219517347085 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/580/statm: 91552 9414 1901 364 0 26775 0
[pid=586] ppid=580 vsize=290428 CPUtime=0 cores=0,2,4,6
/proc/586/stat : 586 (python) S 580 580 27000 0 -1 4202560 894 0 0 0 0 0 0 0 20 0 1 0 338079756 297398272 7869 33554432000 4194304 5683644 140735443741184 140735443732272 219513979304 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/586/statm: 72607 7869 362 364 0 7830 0
[pid=588] ppid=586 vsize=6920 CPUtime=5.09 cores=0,2,4,6
/proc/588/stat : 588 (clasp-1.3.7-x86) R 586 588 588 0 -1 4202496 1920 0 0 0 509 0 0 0 20 0 1 0 338079756 7086080 1458 33554432000 134512640 136139019 4287684720 4287683676 134828656 0 0 16781312 16386 0 0 0 17 0 0 0 0 0 0
/proc/588/statm: 1730 1458 183 398 0 1330 0
[pid=580/tid=587] ppid=578 vsize=366208 CPUtime=0 cores=0,2,4,6
/proc/580/task/587/stat : 587 (python) S 578 580 27000 0 -1 4202560 25 1013 0 0 0 0 0 0 20 0 2 0 338079756 374996992 9414 33554432000 4194304 5683644 140735443741184 140648976749104 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 6.14
Current children cumulated vsize (KiB) 663556

[startup+12.7003 s]
/proc/loadavg: 2.05 2.02 2.00 3/204 588
/proc/meminfo: memFree=25548264/32873648 swapFree=7120/7120
[pid=580] ppid=578 vsize=366208 CPUtime=1.05 cores=0,2,4,6
/proc/580/stat : 580 (python) S 578 580 27000 0 -1 4202496 13949 1013 1 0 96 9 0 0 20 0 2 0 338079636 374996992 9414 33554432000 4194304 5683644 140735443741184 140735443736016 219517347085 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/580/statm: 91552 9414 1901 364 0 26775 0
[pid=586] ppid=580 vsize=290428 CPUtime=0.01 cores=0,2,4,6
/proc/586/stat : 586 (python) S 580 580 27000 0 -1 4202560 901 0 0 0 1 0 0 0 20 0 1 0 338079756 297398272 7870 33554432000 4194304 5683644 140735443741184 140735443732272 219513979304 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/586/statm: 72607 7870 363 364 0 7830 0
[pid=588] ppid=586 vsize=7868 CPUtime=11.48 cores=0,2,4,6
/proc/588/stat : 588 (clasp-1.3.7-x86) R 586 588 588 0 -1 4202496 2152 0 0 0 1148 0 0 0 20 0 1 0 338079756 8056832 1690 33554432000 134512640 136139019 4287684720 4287683696 134883461 0 0 16781312 16386 0 0 0 17 0 0 0 0 0 0
/proc/588/statm: 1967 1690 183 398 0 1567 0
[pid=580/tid=587] ppid=578 vsize=366208 CPUtime=0 cores=0,2,4,6
/proc/580/task/587/stat : 587 (python) S 578 580 27000 0 -1 4202560 25 1013 0 0 0 0 0 0 20 0 2 0 338079756 374996992 9414 33554432000 4194304 5683644 140735443741184 140648976749104 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 12.54
Current children cumulated vsize (KiB) 664504

[startup+25.5003 s]
/proc/loadavg: 2.04 2.02 2.00 3/204 588
/proc/meminfo: memFree=25546156/32873648 swapFree=7120/7120
[pid=580] ppid=578 vsize=366208 CPUtime=1.05 cores=0,2,4,6
/proc/580/stat : 580 (python) S 578 580 27000 0 -1 4202496 13949 1013 1 0 96 9 0 0 20 0 2 0 338079636 374996992 9414 33554432000 4194304 5683644 140735443741184 140735443736016 219517347085 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/580/statm: 91552 9414 1901 364 0 26775 0
[pid=586] ppid=580 vsize=290428 CPUtime=0.03 cores=0,2,4,6
/proc/586/stat : 586 (python) S 580 580 27000 0 -1 4202560 912 0 0 0 3 0 0 0 20 0 1 0 338079756 297398272 7870 33554432000 4194304 5683644 140735443741184 140735443732272 219513979304 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/586/statm: 72607 7870 363 364 0 7830 0
[pid=588] ppid=586 vsize=9336 CPUtime=24.29 cores=0,2,4,6
/proc/588/stat : 588 (clasp-1.3.7-x86) R 586 588 588 0 -1 4202496 2535 0 0 0 2428 1 0 0 20 0 1 0 338079756 9560064 2073 33554432000 134512640 136139019 4287684720 4287683700 134883821 0 0 16781312 16386 0 0 0 17 0 0 0 0 0 0
/proc/588/statm: 2334 2073 183 398 0 1934 0
[pid=580/tid=587] ppid=578 vsize=366208 CPUtime=0 cores=0,2,4,6
/proc/580/task/587/stat : 587 (python) S 578 580 27000 0 -1 4202560 25 1013 0 0 0 0 0 0 20 0 2 0 338079756 374996992 9414 33554432000 4194304 5683644 140735443741184 140648976749104 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 25.37
Current children cumulated vsize (KiB) 665972

[startup+51.1005 s]
/proc/loadavg: 2.02 2.02 2.00 3/208 592
/proc/meminfo: memFree=25531952/32873648 swapFree=7120/7120
[pid=580] ppid=578 vsize=442000 CPUtime=3.04 cores=0,2,4,6
/proc/580/stat : 580 (python) S 578 580 27000 0 -1 4202496 14894 1013 1 0 294 10 0 0 20 0 3 0 338079636 452608000 9423 33554432000 4194304 5683644 140735443741184 140735443736016 219517347085 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/580/statm: 110500 9423 1904 364 0 45720 0
[pid=586] ppid=580 vsize=366208 CPUtime=0.07 cores=0,2,4,6
/proc/586/stat : 586 (python) S 580 580 27000 0 -1 4202560 1153 0 0 0 5 2 0 0 20 0 2 0 338079756 374996992 7910 33554432000 4194304 5683644 140735443741184 140735443732832 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/586/statm: 91552 7910 396 364 0 26775 0
[pid=588] ppid=586 vsize=11184 CPUtime=47.1 cores=0,2,4,6
/proc/588/stat : 588 (clasp-1.3.7-x86) T 586 588 588 0 -1 4202496 2982 0 0 0 4709 1 0 0 20 0 1 0 338079756 11452416 2520 33554432000 134512640 136139019 4287684720 4287683584 134608269 0 0 16781312 16386 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/588/statm: 2796 2520 183 398 0 2396 0
[pid=586/tid=589] ppid=580 vsize=366208 CPUtime=0 cores=0,2,4,6
/proc/586/task/589/stat : 589 (python) S 580 580 27000 0 -1 4202560 42 0 0 0 0 0 0 0 20 0 2 0 338084469 374996992 7910 33554432000 4194304 5683644 140735443741184 140648976749104 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=590] ppid=580 vsize=366220 CPUtime=0 cores=0,2,4,6
/proc/590/stat : 590 (python) S 580 580 27000 0 -1 4202560 768 0 0 0 0 0 0 0 20 0 1 0 338084668 375009280 7848 33554432000 4194304 5683644 140735443741184 140735443732272 219513979363 0 0 16781312 33554946 18446744071580425449 0 0 17 4 0 0 0 0 0
/proc/590/statm: 91555 7848 335 364 0 26775 0
[pid=592] ppid=590 vsize=81892 CPUtime=0.77 cores=0,2,4,6
/proc/592/stat : 592 (minion) R 590 592 592 0 -1 4202496 8039 0 0 0 76 1 0 0 20 0 1 0 338084668 83857408 13108 33554432000 134512640 165565024 4287956016 4287953296 142424342 0 0 16781312 8396802 0 0 0 17 0 0 0 0 0 0
/proc/592/statm: 20473 13108 345 7582 0 12888 0
[pid=580/tid=587] ppid=578 vsize=442000 CPUtime=0 cores=0,2,4,6
/proc/580/task/587/stat : 587 (python) S 578 580 27000 0 -1 4202560 25 1013 0 0 0 0 0 0 20 0 3 0 338079756 452608000 9423 33554432000 4194304 5683644 140735443741184 140648976749104 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=580/tid=591] ppid=578 vsize=442000 CPUtime=0 cores=0,2,4,6
/proc/580/task/591/stat : 591 (python) S 578 580 27000 0 -1 4202560 22 1013 0 0 0 0 0 0 20 0 3 0 338084668 452608000 9423 33554432000 4194304 5683644 140735443741184 140648966259248 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 50.98
Current children cumulated vsize (KiB) 1267504

[startup+102.307 s]
/proc/loadavg: 2.01 2.01 2.00 3/212 596
/proc/meminfo: memFree=25467952/32873648 swapFree=7120/7120
[pid=580] ppid=578 vsize=517792 CPUtime=3.06 cores=0,2,4,6
/proc/580/stat : 580 (python) S 578 580 27000 0 -1 4202496 15772 1013 1 0 296 10 0 0 20 0 4 0 338079636 530219008 9431 33554432000 4194304 5683644 140735443741184 140735443736016 219517347085 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/580/statm: 129448 9431 1906 364 0 64665 0
[pid=586] ppid=580 vsize=366208 CPUtime=0.07 cores=0,2,4,6
/proc/586/stat : 586 (python) S 580 580 27000 0 -1 4202560 1153 0 0 0 5 2 0 0 20 0 2 0 338079756 374996992 7910 33554432000 4194304 5683644 140735443741184 140735443732832 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/586/statm: 91552 7910 396 364 0 26775 0
[pid=588] ppid=586 vsize=11184 CPUtime=47.1 cores=0,2,4,6
/proc/588/stat : 588 (clasp-1.3.7-x86) T 586 588 588 0 -1 4202496 2982 0 0 0 4709 1 0 0 20 0 1 0 338079756 11452416 2520 33554432000 134512640 136139019 4287684720 4287683584 134608269 0 0 16781312 16386 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/588/statm: 2796 2520 183 398 0 2396 0
[pid=586/tid=589] ppid=580 vsize=366208 CPUtime=0 cores=0,2,4,6
/proc/586/task/589/stat : 589 (python) S 580 580 27000 0 -1 4202560 42 0 0 0 0 0 0 0 20 0 2 0 338084469 374996992 7910 33554432000 4194304 5683644 140735443741184 140648976749104 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=590] ppid=580 vsize=366220 CPUtime=0.08 cores=0,2,4,6
/proc/590/stat : 590 (python) S 580 580 27000 0 -1 4202560 1067 0 0 0 5 3 0 0 20 0 2 0 338084668 375009280 7897 33554432000 4194304 5683644 140735443741184 140735443732832 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/590/statm: 91555 7897 383 364 0 26775 0
[pid=592] ppid=590 vsize=81892 CPUtime=47.71 cores=0,2,4,6
/proc/592/stat : 592 (minion) T 590 592 592 0 -1 4202496 8039 0 0 0 4770 1 0 0 20 0 1 0 338084668 83857408 13108 33554432000 134512640 165565024 4287956016 4287953216 134640658 0 0 16781312 8396802 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/592/statm: 20473 13108 345 7582 0 12888 0
[pid=590/tid=593] ppid=580 vsize=366220 CPUtime=0 cores=0,2,4,6
/proc/590/task/593/stat : 593 (python) S 580 580 27000 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 2 0 338089442 375009280 7897 33554432000 4194304 5683644 140735443741184 140648976749104 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=594] ppid=580 vsize=442012 CPUtime=0 cores=0,2,4,6
/proc/594/stat : 594 (python) S 580 580 27000 0 -1 4202560 920 0 0 0 0 0 0 0 20 0 1 0 338089444 452620288 7877 33554432000 4194304 5683644 140735443741184 140735443732272 219513979363 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/594/statm: 110503 7877 358 364 0 45720 0
[pid=596] ppid=594 vsize=14216 CPUtime=4.21 cores=0,2,4,6
/proc/596/stat : 596 (wbo1.4b-fixed) R 594 596 596 0 -1 4202496 3649 0 0 0 418 3 0 0 20 0 1 0 338089444 14557184 3064 33554432000 134512640 135752315 4287290384 4287288392 134519014 0 0 16781312 3 0 0 0 17 0 0 0 0 0 0
/proc/596/statm: 3554 3064 146 303 0 3249 0
[pid=580/tid=587] ppid=578 vsize=517792 CPUtime=0 cores=0,2,4,6
/proc/580/task/587/stat : 587 (python) S 578 580 27000 0 -1 4202560 25 1013 0 0 0 0 0 0 20 0 4 0 338079756 530219008 9431 33554432000 4194304 5683644 140735443741184 140648976749104 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=580/tid=591] ppid=578 vsize=517792 CPUtime=0 cores=0,2,4,6
/proc/580/task/591/stat : 591 (python) S 578 580 27000 0 -1 4202560 22 1013 0 0 0 0 0 0 20 0 4 0 338084668 530219008 9431 33554432000 4194304 5683644 140735443741184 140648966259248 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0

################
# More data... #
################

/proc/586/stat : 586 (python) S 580 580 27000 0 -1 4202560 1216 0 0 0 24 14 0 0 20 0 2 0 338079756 374996992 7911 33554432000 4194304 5683644 140735443741184 140735443732832 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/586/statm: 91552 7911 397 364 0 26775 0
[pid=588] ppid=586 vsize=19112 CPUtime=229.54 cores=0,2,4,6
/proc/588/stat : 588 (clasp-1.3.7-x86) T 586 588 588 0 -1 4202496 4961 0 0 0 22953 1 0 0 20 0 1 0 338079756 19570688 4503 33554432000 134512640 136139019 4287684720 4287683696 134883415 0 0 16781312 16386 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/588/statm: 4778 4503 183 398 0 4378 0
[pid=586/tid=589] ppid=580 vsize=366208 CPUtime=0 cores=0,2,4,6
/proc/586/task/589/stat : 589 (python) S 580 580 27000 0 -1 4202560 42 0 0 0 0 0 0 0 20 0 2 0 338084469 374996992 7911 33554432000 4194304 5683644 140735443741184 140648976749104 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=590] ppid=580 vsize=366220 CPUtime=0.08 cores=0,2,4,6
/proc/590/stat : 590 (python) S 580 580 27000 0 -1 4202560 1067 0 0 0 5 3 0 0 20 0 2 0 338084668 375009280 7897 33554432000 4194304 5683644 140735443741184 140735443732832 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/590/statm: 91555 7897 383 364 0 26775 0
[pid=592] ppid=590 vsize=81892 CPUtime=47.71 cores=0,2,4,6
/proc/592/stat : 592 (minion) T 590 592 592 0 -1 4202496 8039 0 0 0 4770 1 0 0 20 0 1 0 338084668 83857408 13156 33554432000 134512640 165565024 4287956016 4287953216 134640658 0 0 16781312 8396802 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/592/statm: 20473 13156 345 7582 0 12888 0
[pid=590/tid=593] ppid=580 vsize=366220 CPUtime=0 cores=0,2,4,6
/proc/590/task/593/stat : 593 (python) S 580 580 27000 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 2 0 338089442 375009280 7897 33554432000 4194304 5683644 140735443741184 140648976749104 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=594] ppid=580 vsize=442012 CPUtime=0.07 cores=0,2,4,6
/proc/594/stat : 594 (python) S 580 580 27000 0 -1 4202560 1106 0 0 0 4 3 0 0 20 0 2 0 338089444 452620288 7903 33554432000 4194304 5683644 140735443741184 140735443732832 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/594/statm: 110503 7903 383 364 0 45720 0
[pid=596] ppid=594 vsize=46548 CPUtime=49.1 cores=0,2,4,6
/proc/596/stat : 596 (wbo1.4b-fixed) T 594 596 596 0 -1 4202496 11692 0 0 0 4886 24 0 0 20 0 1 0 338089444 47665152 11121 33554432000 134512640 135752315 4287290384 4287287904 134638604 0 0 16781312 3 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/596/statm: 11637 11121 147 303 0 11332 0
[pid=594/tid=642] ppid=580 vsize=442012 CPUtime=0 cores=0,2,4,6
/proc/594/task/642/stat : 642 (python) S 580 580 27000 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 2 0 338094356 452620288 7903 33554432000 4194304 5683644 140735443741184 140648966259248 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=643] ppid=580 vsize=517804 CPUtime=0.07 cores=0,2,4,6
/proc/643/stat : 643 (python) S 580 580 27000 0 -1 4202560 1102 0 0 0 5 2 0 0 20 0 2 0 338094359 530231296 7909 33554432000 4194304 5683644 140735443741184 140735443732832 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/643/statm: 129451 7909 383 364 0 64665 0
[pid=645] ppid=643 vsize=190492 CPUtime=48.91 cores=0,2,4,6
/proc/645/stat : 645 (scip-2.0.1.linu) T 643 645 645 0 -1 4202496 46229 0 0 0 4863 28 0 0 20 0 1 0 338094359 195063808 44877 33554432000 4194304 13684479 140734553747472 140734553739608 7318405 0 0 16781312 2 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/645/statm: 47623 44877 772 2317 0 45301 0
[pid=643/tid=650] ppid=580 vsize=517804 CPUtime=0 cores=0,2,4,6
/proc/643/task/650/stat : 650 (python) S 580 580 27000 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 2 0 338099253 530231296 7909 33554432000 4194304 5683644 140735443741184 140648955769392 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=653] ppid=580 vsize=593596 CPUtime=0.07 cores=0,2,4,6
/proc/653/stat : 653 (python) S 580 580 27000 0 -1 4202560 1119 0 0 0 4 3 0 0 20 0 2 0 338103770 607842304 7915 33554432000 4194304 5683644 140735443741184 140735443732832 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/653/statm: 148399 7915 383 364 0 83610 0
[pid=655] ppid=653 vsize=49316 CPUtime=48.74 cores=0,2,4,6
/proc/655/stat : 655 (wbo1.4a) T 653 655 655 0 -1 4202496 12406 0 0 0 4853 21 0 0 20 0 1 0 338103770 50499584 11817 33554432000 134512640 135751879 4291326896 4291325072 134616371 0 0 16781312 3 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/655/statm: 12329 11817 143 303 0 12024 0
[pid=653/tid=657] ppid=580 vsize=593596 CPUtime=0 cores=0,2,4,6
/proc/653/task/657/stat : 657 (python) S 580 580 27000 0 -1 4202560 46 0 0 0 0 0 0 0 20 0 2 0 338108647 607842304 7915 33554432000 4194304 5683644 140735443741184 140648945279536 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=658] ppid=580 vsize=669388 CPUtime=0.07 cores=0,2,4,6
/proc/658/stat : 658 (python) S 580 580 27000 0 -1 4202560 1128 0 0 0 4 3 0 0 20 0 2 0 338108649 685453312 7921 33554432000 4194304 5683644 140735443741184 140735443732832 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/658/statm: 167347 7921 383 364 0 102555 0
[pid=660] ppid=658 vsize=9624972 CPUtime=49.21 cores=0,2,4,6
/proc/660/stat : 660 (java) T 658 660 660 0 -1 4202496 16310 0 1 0 4906 15 0 0 20 0 19 0 338108649 9855971328 34408 33554432000 1073741824 1073778376 140735171950624 140735171941760 219517321277 0 0 0 16800975 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/660/statm: 2406243 34408 2278 9 0 2396937 0
[pid=660/tid=661] ppid=658 vsize=9624972 CPUtime=46.68 cores=0,2,4,6
/proc/660/task/661/stat : 661 (java) T 658 660 660 0 -1 4202560 2579 0 1 0 4660 8 0 0 20 0 19 0 338108651 9855971328 34408 33554432000 1073741824 1073778376 140735171950624 139667575278864 139667448660567 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=660/tid=662] ppid=658 vsize=9624972 CPUtime=0.08 cores=0,2,4,6
/proc/660/task/662/stat : 662 (java) T 658 660 660 0 -1 4202560 32 0 0 0 8 0 0 0 20 0 19 0 338108652 9855971328 34408 33554432000 1073741824 1073778376 140735171950624 139667419249192 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=660/tid=663] ppid=658 vsize=9624972 CPUtime=0.09 cores=0,2,4,6
/proc/660/task/663/stat : 663 (java) T 658 660 660 0 -1 4202560 24 0 0 0 9 0 0 0 20 0 19 0 338108652 9855971328 34408 33554432000 1073741824 1073778376 140735171950624 139667418196392 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=660/tid=664] ppid=658 vsize=9624972 CPUtime=0.09 cores=0,2,4,6
/proc/660/task/664/stat : 664 (java) T 658 660 660 0 -1 4202560 15 0 0 0 9 0 0 0 20 0 19 0 338108652 9855971328 34408 33554432000 1073741824 1073778376 140735171950624 139667417143592 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=660/tid=665] ppid=658 vsize=9624972 CPUtime=0.1 cores=0,2,4,6
/proc/660/task/665/stat : 665 (java) T 658 660 660 0 -1 4202560 26 0 0 0 10 0 0 0 20 0 19 0 338108652 9855971328 34408 33554432000 1073741824 1073778376 140735171950624 139667416090792 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=660/tid=666] ppid=658 vsize=9624972 CPUtime=0.07 cores=0,2,4,6
/proc/660/task/666/stat : 666 (java) T 658 660 660 0 -1 4202560 15 0 0 0 7 0 0 0 20 0 19 0 338108652 9855971328 34408 33554432000 1073741824 1073778376 140735171950624 139667415039016 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=660/tid=667] ppid=658 vsize=9624972 CPUtime=0.1 cores=0,2,4,6
/proc/660/task/667/stat : 667 (java) T 658 660 660 0 -1 4202560 17 0 0 0 10 0 0 0 20 0 19 0 338108652 9855971328 34408 33554432000 1073741824 1073778376 140735171950624 139667413986216 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=660/tid=668] ppid=658 vsize=9624972 CPUtime=0.08 cores=0,2,4,6
/proc/660/task/668/stat : 668 (java) T 658 660 660 0 -1 4202560 9 0 0 0 8 0 0 0 20 0 19 0 338108652 9855971328 34408 33554432000 1073741824 1073778376 140735171950624 139667412933416 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=660/tid=669] ppid=658 vsize=9624972 CPUtime=0.09 cores=0,2,4,6
/proc/660/task/669/stat : 669 (java) T 658 660 660 0 -1 4202560 24 0 0 0 9 0 0 0 20 0 19 0 338108652 9855971328 34408 33554432000 1073741824 1073778376 140735171950624 139667411880616 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=660/tid=670] ppid=658 vsize=9624972 CPUtime=0.37 cores=0,2,4,6
/proc/660/task/670/stat : 670 (java) T 658 660 660 0 -1 4202560 66 0 0 0 35 2 0 0 20 0 19 0 338108653 9855971328 34408 33554432000 1073741824 1073778376 140735171950624 139667399469544 219517335465 0 0 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=660/tid=671] ppid=658 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/660/task/671/stat : 671 (java) T 658 660 660 0 -1 4202560 17 0 0 0 0 0 0 0 20 0 19 0 338108653 9855971328 34408 33554432000 1073741824 1073778376 140735171950624 139667398415608 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=660/tid=672] ppid=658 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/660/task/672/stat : 672 (java) T 658 660 660 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 19 0 338108653 9855971328 34408 33554432000 1073741824 1073778376 140735171950624 139667397362696 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=660/tid=673] ppid=658 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/660/task/673/stat : 673 (java) T 658 660 660 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338108656 9855971328 34408 33554432000 1073741824 1073778376 140735171950624 139667396311184 219517343520 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=660/tid=674] ppid=658 vsize=9624972 CPUtime=0.66 cores=0,2,4,6
/proc/660/task/674/stat : 674 (java) T 658 660 660 0 -1 4202560 6342 0 0 0 64 2 0 0 20 0 19 0 338108656 9855971328 34408 33554432000 1073741824 1073778376 140735171950624 139667395259320 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=660/tid=675] ppid=658 vsize=9624972 CPUtime=0.65 cores=0,2,4,6
/proc/660/task/675/stat : 675 (java) T 658 660 660 0 -1 4202560 5740 0 0 0 63 2 0 0 20 0 19 0 338108656 9855971328 34408 33554432000 1073741824 1073778376 140735171950624 139667394206520 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=660/tid=676] ppid=658 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/660/task/676/stat : 676 (java) T 658 660 660 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338108656 9855971328 34408 33554432000 1073741824 1073778376 140735171950624 139667393153912 219517334588 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=660/tid=677] ppid=658 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/660/task/677/stat : 677 (java) T 658 660 660 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 19 0 338108656 9855971328 34408 33554432000 1073741824 1073778376 140735171950624 139667392101320 219517335465 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=660/tid=678] ppid=658 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/660/task/678/stat : 678 (java) T 658 660 660 0 -1 4202560 12 0 0 0 0 0 0 0 20 0 19 0 338108711 9855971328 34408 33554432000 1073741824 1073778376 140735171950624 139667391046824 219517335465 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=658/tid=684] ppid=580 vsize=669388 CPUtime=0 cores=0,2,4,6
/proc/658/task/684/stat : 684 (python) S 580 580 27000 0 -1 4202560 46 0 0 0 0 0 0 0 20 0 2 0 338113389 685453312 7921 33554432000 4194304 5683644 140735443741184 140648934789680 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=685] ppid=580 vsize=745180 CPUtime=0.07 cores=0,2,4,6
/proc/685/stat : 685 (python) S 580 580 27000 0 -1 4202560 1187 0 0 0 4 3 0 0 20 0 2 0 338113392 763064320 7933 33554432000 4194304 5683644 140735443741184 140735443732832 219517347085 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/685/statm: 186295 7933 389 364 0 121500 0
[pid=687] ppid=685 vsize=200232 CPUtime=46.94 cores=0,2,4,6
/proc/687/stat : 687 (scip-2.0.1.linu) T 685 687 687 0 -1 4202496 70133 0 0 0 4654 40 0 0 20 0 1 0 338113392 205037568 45627 33554432000 4194304 15243407 140734383387680 140734383382824 8135920 0 0 16781312 2 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/687/statm: 50058 45627 915 2698 0 47353 0
[pid=685/tid=692] ppid=580 vsize=745180 CPUtime=0 cores=0,2,4,6
/proc/685/task/692/stat : 692 (python) S 580 580 27000 0 -1 4202560 101 0 0 0 0 0 0 0 20 0 2 0 338118089 763064320 7933 33554432000 4194304 5683644 140735443741184 140648924299824 219517343520 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=704] ppid=580 vsize=820972 CPUtime=0 cores=0,2,4,6
/proc/704/stat : 704 (python) S 580 580 27000 0 -1 4202560 850 0 0 0 0 0 0 0 20 0 1 0 338131833 840675328 8392 33554432000 4194304 5683644 140735443741184 140735443732272 219513979363 0 0 16781312 33554946 18446744071580425449 0 0 17 6 0 0 0 0 0
/proc/704/statm: 205243 8392 335 364 0 140445 0
[pid=706] ppid=704 vsize=9624972 CPUtime=2.38 cores=0,2,4,6
/proc/706/stat : 706 (java) S 704 706 706 0 -1 4202496 19763 0 1 0 232 6 0 0 20 0 19 0 338131833 9855971328 44031 33554432000 1073741824 1073778376 140737290614640 140737290605776 219517321277 0 0 0 16800975 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/706/statm: 2406243 44031 2265 9 0 2396937 0
[pid=706/tid=707] ppid=704 vsize=9624972 CPUtime=1 cores=0,2,4,6
/proc/706/task/707/stat : 707 (java) R 704 706 706 0 -1 4202560 2577 0 1 0 97 3 0 0 20 0 19 0 338131834 9855971328 44031 33554432000 1073741824 1073778376 140737290614640 139695766703000 139695639573421 0 4 0 16800975 0 0 0 -1 0 0 0 0 0 0
[pid=706/tid=708] ppid=704 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/706/task/708/stat : 708 (java) S 704 706 706 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 19 0 338131835 9855971328 44031 33554432000 1073741824 1073778376 140737290614640 139695610526504 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=706/tid=709] ppid=704 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/706/task/709/stat : 709 (java) S 704 706 706 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338131835 9855971328 44031 33554432000 1073741824 1073778376 140737290614640 139695609473704 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=706/tid=710] ppid=704 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/706/task/710/stat : 710 (java) S 704 706 706 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338131835 9855971328 44031 33554432000 1073741824 1073778376 140737290614640 139695608421416 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=706/tid=711] ppid=704 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/706/task/711/stat : 711 (java) S 704 706 706 0 -1 4202560 11 0 0 0 0 0 0 0 20 0 19 0 338131835 9855971328 44031 33554432000 1073741824 1073778376 140737290614640 139695607368616 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=706/tid=712] ppid=704 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/706/task/712/stat : 712 (java) S 704 706 706 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338131835 9855971328 44031 33554432000 1073741824 1073778376 140737290614640 139695606315304 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=706/tid=713] ppid=704 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/706/task/713/stat : 713 (java) S 704 706 706 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 19 0 338131835 9855971328 44031 33554432000 1073741824 1073778376 140737290614640 139695605262504 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=706/tid=714] ppid=704 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/706/task/714/stat : 714 (java) S 704 706 706 0 -1 4202560 5 0 0 0 0 0 0 0 20 0 19 0 338131835 9855971328 44031 33554432000 1073741824 1073778376 140737290614640 139695604210216 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=706/tid=716] ppid=704 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/706/task/716/stat : 716 (java) S 704 706 706 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 19 0 338131835 9855971328 44031 33554432000 1073741824 1073778376 140737290614640 139695603157416 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=706/tid=717] ppid=704 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/706/task/717/stat : 717 (java) S 704 706 706 0 -1 4202560 9 0 0 0 0 0 0 0 20 0 19 0 338131836 9855971328 44031 33554432000 1073741824 1073778376 140737290614640 139695590959848 219517335465 0 0 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=706/tid=718] ppid=704 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/706/task/718/stat : 718 (java) S 704 706 706 0 -1 4202560 17 0 0 0 0 0 0 0 20 0 19 0 338131837 9855971328 44031 33554432000 1073741824 1073778376 140737290614640 139695589905912 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=706/tid=719] ppid=704 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/706/task/719/stat : 719 (java) S 704 706 706 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 19 0 338131837 9855971328 44031 33554432000 1073741824 1073778376 140737290614640 139695588853512 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=706/tid=720] ppid=704 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/706/task/720/stat : 720 (java) S 704 706 706 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338131839 9855971328 44031 33554432000 1073741824 1073778376 140737290614640 139695587802000 219517343520 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=706/tid=721] ppid=704 vsize=9624972 CPUtime=0.63 cores=0,2,4,6
/proc/706/task/721/stat : 721 (java) S 704 706 706 0 -1 4202560 5720 0 0 0 62 1 0 0 20 0 19 0 338131839 9855971328 44031 33554432000 1073741824 1073778376 140737290614640 139695586748600 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=706/tid=722] ppid=704 vsize=9624972 CPUtime=0.68 cores=0,2,4,6
/proc/706/task/722/stat : 722 (java) S 704 706 706 0 -1 4202560 10003 0 0 0 67 1 0 0 20 0 19 0 338131839 9855971328 44031 33554432000 1073741824 1073778376 140737290614640 139695585695800 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=706/tid=723] ppid=704 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/706/task/723/stat : 723 (java) S 704 706 706 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338131839 9855971328 44031 33554432000 1073741824 1073778376 140737290614640 139695584643704 219517334588 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=706/tid=724] ppid=704 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/706/task/724/stat : 724 (java) S 704 706 706 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338131839 9855971328 44031 33554432000 1073741824 1073778376 140737290614640 139695583591112 219517335465 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=706/tid=725] ppid=704 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/706/task/725/stat : 725 (java) S 704 706 706 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 19 0 338131896 9855971328 44031 33554432000 1073741824 1073778376 140737290614640 139695582537128 219517335465 0 4 0 16800975 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=580/tid=587] ppid=578 vsize=896752 CPUtime=0 cores=0,2,4,6
/proc/580/task/587/stat : 587 (python) S 578 580 27000 0 -1 4202560 33 1013 0 0 0 0 0 0 20 0 9 0 338079756 918274048 9979 33554432000 4194304 5683644 140735443741184 140648976749104 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=580/tid=591] ppid=578 vsize=896752 CPUtime=0 cores=0,2,4,6
/proc/580/task/591/stat : 591 (python) S 578 580 27000 0 -1 4202560 22 1013 0 0 0 0 0 0 20 0 9 0 338084668 918274048 9979 33554432000 4194304 5683644 140735443741184 140648966259248 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=580/tid=595] ppid=578 vsize=896752 CPUtime=0 cores=0,2,4,6
/proc/580/task/595/stat : 595 (python) S 578 580 27000 0 -1 4202560 22 1013 0 0 0 0 0 0 20 0 9 0 338089444 918274048 9979 33554432000 4194304 5683644 140735443741184 140648955769392 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=580/tid=644] ppid=578 vsize=896752 CPUtime=0 cores=0,2,4,6
/proc/580/task/644/stat : 644 (python) S 578 580 27000 0 -1 4202560 21 1013 0 0 0 0 0 0 20 0 9 0 338094359 918274048 9979 33554432000 4194304 5683644 140735443741184 140648945279536 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=580/tid=654] ppid=578 vsize=896752 CPUtime=0 cores=0,2,4,6
/proc/580/task/654/stat : 654 (python) S 578 580 27000 0 -1 4202560 22 1013 0 0 0 0 0 0 20 0 9 0 338103770 918274048 9979 33554432000 4194304 5683644 140735443741184 140648934789680 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=580/tid=659] ppid=578 vsize=896752 CPUtime=0 cores=0,2,4,6
/proc/580/task/659/stat : 659 (python) S 578 580 27000 0 -1 4202560 21 1013 0 0 0 0 0 0 20 0 9 0 338108649 918274048 9979 33554432000 4194304 5683644 140735443741184 140648924299824 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=580/tid=686] ppid=578 vsize=896752 CPUtime=0 cores=0,2,4,6
/proc/580/task/686/stat : 686 (python) S 578 580 27000 0 -1 4202560 20 1013 0 0 0 0 0 0 20 0 9 0 338113392 918274048 9979 33554432000 4194304 5683644 140735443741184 140648569102896 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=580/tid=705] ppid=578 vsize=896752 CPUtime=0 cores=0,2,4,6
/proc/580/task/705/stat : 705 (python) S 578 580 27000 0 -1 4202560 22 1013 0 0 0 0 0 0 20 0 9 0 338131833 918274048 9979 33554432000 4194304 5683644 140735443741184 140648558613040 219517343520 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 526.59
Current children cumulated vsize (KiB) 25255668

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 580 and gives
#  childrusage.ru_utime.tv_sec=3
#  childrusage.ru_utime.tv_usec=135523
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=125980
# CPU time returned by wait4() is 3.2615
# while last known CPU time is 526.59
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 523.071
CPU time (s): 526.59
CPU user time (s): 524.8
CPU system time (s): 1.79
CPU usage (%): 100.673
Max. virtual memory (cumulated for all children) (KiB): 25255668

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 3.13552
system time used= 0.12598
maximum resident set size= 39916
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 21603
page faults= 1
swaps= 0
block input operations= 0
block output operations= 4304
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1028
involuntary context switches= 32

runsolver used 2.43263 second user time and 5.21321 second system time

The end

Launcher Data 

Begin job on node114 at 2012-06-03 12:32:02
IDJOB=3740074
IDBENCH=76228
IDSOLVER=2327
FILE ID=node114/3740074-1338719522
RUNJOBID= node114-1338713908-27016
PBS_JOBID= 14636748
Free space on /tmp= 70036 MiB

SOLVER NAME= PB11: borg pb-dec-11.04.03
BENCH NAME= PB10/normalized-PB10/DEC-SMALLINT-LIN/oliveras/j60/normalized-j6029_8-unsat.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.dec.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3740074-1338719522/watcher-3740074-1338719522 -o /tmp/evaluation-result-3740074-1338719522/solver-3740074-1338719522 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3740074-1338719522.opb -seed 1611718967 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= c619d5d1f14c9c14bda19749a65da87b
RANDOM SEED=1611718967

node114.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.48
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5451.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.242
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873648 kB
MemFree:        25586472 kB
Buffers:         1032812 kB
Cached:          4499660 kB
SwapCached:         2312 kB
Active:          3782904 kB
Inactive:        2509572 kB
Active(anon):     746448 kB
Inactive(anon):     8516 kB
Active(file):    3036456 kB
Inactive(file):  2501056 kB
Unevictable:        7120 kB
Mlocked:            7120 kB
SwapTotal:      67108856 kB
SwapFree:       67103580 kB
Dirty:            107888 kB
Writeback:             0 kB
AnonPages:        757728 kB
Mapped:            27500 kB
Shmem:               188 kB
Slab:             850416 kB
SReclaimable:     291212 kB
SUnreclaim:       559204 kB
KernelStack:        1640 kB
PageTables:         9788 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545680 kB
Committed_AS:    1324232 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346400 kB
VmallocChunk:   34341988052 kB
HardwareCorrupted:     0 kB
AnonHugePages:    135168 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7680 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 66148 MiB
End job on node114 at 2012-06-03 12:40:48