Trace number 3738043

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB11: borg pb-dec-11.04.03? (MO) 1670.57 1665.97

General information on the benchmark

Namenormalized-PB07/SATUNSAT-SMALLINT-NLC/submittedPB07/
manquinho/dbsg/normalized-dbsg_100_25_2_15.opb
MD5SUMb437bd5db918ab114396d43dfca73285
Bench CategoryDEC-SMALLINT-NLC (no optimisation, small integers, non linear constraints)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark0
Best CPU time to get the best result obtained on this benchmark93.6968
Has Objective FunctionNO
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function
Optimality of the best value was proved NO
Number of variables200
Total number of constraints302
Number of constraints which are clauses100
Number of constraints which are cardinality constraints (but not clauses)1
Number of constraints which are nor clauses,nor cardinality constraints201
Minimum length of a constraint2
Maximum length of a constraint200
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 15
Number of bits of the biggest number in a constraint 4
Biggest sum of numbers in a constraint 200
Number of bits of the biggest sum of numbers8
Number of products (including duplicates)6148
Sum of products size (including duplicates)12296
Number of different products3074
Sum of products size6148

Solver Data 

0.40/0.51	c INFO: loaded portfolio from HOME/etc/borg-mix+class.dec.pickle
0.40/0.51	c INFO: solving HOME/instance-3738043-1338717741.opb
0.40/0.56	c INFO: wrote linearized instance to HOME/tmpRhMgYA.opb
0.50/0.61	c INFO: parsing took 0.05 s
0.50/0.69	c INFO: features took 0.08 CPU seconds
0.60/0.72	c INFO: running clasp-1.3.7@0 for 45 with 1799 remaining (b = 0.51)
47.73/47.88	c INFO: running scip-2.0.1-spx@0 for 45 with 1752 remaining (b = 0.13)
95.87/95.99	c INFO: running scip-2.0.1-clp@0 for 45 with 1704 remaining (b = 0.03)
142.31/142.40	c INFO: running clasp-1.3.7@47 for 45 with 1658 remaining (b = 0.01)
187.98/187.93	c INFO: wrote minion input file to HOME/tmpdv1JrX.minion
187.98/187.93	c INFO: running minion-0.12@0 for 45 with 1613 remaining (b = 0.02)
235.33/235.24	c INFO: running scip-2.0.1-clp@46 for 45 with 1565 remaining (b = 0.01)
282.59/282.50	c INFO: running pbct-0.1.2-linear@0 for 45 with 1518 remaining (b = 0.01)
329.83/329.69	c INFO: running clasp-1.3.7@92 for 136 with 1472 remaining (b = 0.02)
467.44/467.09	c INFO: running scip-2.0.1-spx@47 for 45 with 1334 remaining (b = 0.01)
515.84/515.32	c INFO: running sat4j-pb-v20101225-cutting@0 for 45 with 1287 remaining (b = 0.03)
561.82/558.88	c INFO: running scip-2.0.1-spx@95 for 410 with 1241 remaining (b = 0.06)
976.25/972.70	c INFO: running bsolo_pb10-l3@0 for 45 with 831 remaining (b = 0.01)
1024.40/1020.78	c INFO: running bsolo_pb10-l1@0 for 91 with 783 remaining (b = 0.01)
1116.73/1112.99	c INFO: running scip-2.0.1-spx@505 for 547 with 691 remaining (b = 0.05)
1669.58/1664.91	c INFO: running wbo1.4a@0 for 45 with 143 remaining (b = 0.00)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3738043-1338717741/watcher-3738043-1338717741 -o /tmp/evaluation-result-3738043-1338717741/solver-3738043-1338717741 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3738043-1338717741.opb -seed 1276299188 -budget 1800 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.18 2.08 2.09 2/218 5388
/proc/meminfo: memFree=28644028/32873844 swapFree=7116/7116
[pid=5388] ppid=5386 vsize=15412 CPUtime=0 cores=0,2,4,6
/proc/5388/stat : 5388 (python) R 5386 5388 2614 0 -1 4202496 749 0 0 0 0 0 0 0 20 0 1 0 337899095 15781888 627 33554432000 4194304 5683644 140733698290352 140733698248808 4620223 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/5388/statm: 3853 627 333 364 0 295 0

[startup+0.0526031 s]
/proc/loadavg: 2.18 2.08 2.09 2/218 5388
/proc/meminfo: memFree=28644028/32873844 swapFree=7116/7116
[pid=5388] ppid=5386 vsize=29988 CPUtime=0.03 cores=0,2,4,6
/proc/5388/stat : 5388 (python) R 5386 5388 2614 0 -1 4202496 1531 0 0 0 3 0 0 0 20 0 1 0 337899095 30707712 1269 33554432000 4194304 5683644 140733698290352 140733698244072 240565175184 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/5388/statm: 7497 1269 452 364 0 822 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 29988

[startup+0.100224 s]
/proc/loadavg: 2.18 2.08 2.09 2/218 5388
/proc/meminfo: memFree=28644028/32873844 swapFree=7116/7116
[pid=5388] ppid=5386 vsize=92392 CPUtime=0.07 cores=0,2,4,6
/proc/5388/stat : 5388 (python) R 5386 5388 2614 0 -1 4202496 3221 504 1 0 5 2 0 0 20 0 1 0 337899095 94609408 2541 33554432000 4194304 5683644 140733698290352 140733698242824 240564796704 0 0 16781312 2 0 0 0 17 2 0 0 0 0 0
/proc/5388/statm: 23098 2541 816 364 0 1620 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 92392

[startup+0.300217 s]
/proc/loadavg: 2.18 2.08 2.09 2/218 5388
/proc/meminfo: memFree=28644028/32873844 swapFree=7116/7116
[pid=5388] ppid=5386 vsize=166944 CPUtime=0.24 cores=0,2,4,6
/proc/5388/stat : 5388 (python) R 5386 5388 2614 0 -1 4202496 6561 1009 1 0 18 6 0 0 20 0 1 0 337899095 170950656 4351 33554432000 4194304 5683644 140733698290352 140733698194584 140436939109829 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/5388/statm: 41736 4351 1227 364 0 3289 0
Current children cumulated CPU time (s) 0.24
Current children cumulated vsize (KiB) 166944

[startup+0.700216 s]
/proc/loadavg: 2.18 2.08 2.09 2/218 5388
/proc/meminfo: memFree=28644028/32873844 swapFree=7116/7116
[pid=5388] ppid=5386 vsize=283452 CPUtime=0.6 cores=0,2,4,6
/proc/5388/stat : 5388 (python) R 5386 5388 2614 0 -1 4202496 11498 1417 1 0 46 11 3 0 20 0 1 0 337899095 290254848 7620 33554432000 4194304 5683644 140733698290352 140733698270136 4502173 0 0 16781312 33554434 0 0 0 17 0 0 0 0 0 0
/proc/5388/statm: 70863 7620 1860 364 0 6092 0
Current children cumulated CPU time (s) 0.6
Current children cumulated vsize (KiB) 283452

[startup+1.50028 s]
/proc/loadavg: 2.18 2.08 2.09 3/222 5397
/proc/meminfo: memFree=28616468/32873844 swapFree=7116/7116
[pid=5388] ppid=5386 vsize=359224 CPUtime=0.62 cores=0,2,4,6
/proc/5388/stat : 5388 (python) S 5386 5388 2614 0 -1 4202496 11923 1417 1 0 48 11 3 0 20 0 2 0 337899095 367845376 7669 33554432000 4194304 5683644 140733698290352 140733698285184 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/5388/statm: 89806 7669 1902 364 0 25029 0
[pid=5395] ppid=5388 vsize=283444 CPUtime=0 cores=0,2,4,6
/proc/5395/stat : 5395 (python) S 5388 5388 2614 0 -1 4202560 679 0 0 0 0 0 0 0 20 0 1 0 337899167 290246656 6065 33554432000 4194304 5683644 140733698290352 140733698281440 240565191080 0 0 16781312 33554946 18446744071580425449 0 0 17 4 0 0 0 0 0
/proc/5395/statm: 70861 6065 304 364 0 6084 0
[pid=5397] ppid=5395 vsize=4480 CPUtime=0.77 cores=0,2,4,6
/proc/5397/stat : 5397 (clasp-1.3.7-x86) R 5395 5397 5397 0 -1 4202496 1133 0 0 0 77 0 0 0 20 0 1 0 337899167 4587520 865 33554432000 134512640 136139019 4290128384 4290127296 134619944 0 0 16781312 16386 0 0 0 17 0 0 0 0 0 0
/proc/5397/statm: 1120 865 182 398 0 720 0
[pid=5388/tid=5396] ppid=5386 vsize=359224 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5396/stat : 5396 (python) S 5386 5388 2614 0 -1 4202560 27 1417 0 0 0 0 3 0 20 0 2 0 337899167 367845376 7669 33554432000 4194304 5683644 140733698290352 140436765745712 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1.39
Current children cumulated vsize (KiB) 647148

[startup+3.10028 s]
/proc/loadavg: 2.18 2.08 2.09 3/222 5397
/proc/meminfo: memFree=28610872/32873844 swapFree=7116/7116
[pid=5388] ppid=5386 vsize=359224 CPUtime=0.62 cores=0,2,4,6
/proc/5388/stat : 5388 (python) S 5386 5388 2614 0 -1 4202496 11923 1417 1 0 48 11 3 0 20 0 2 0 337899095 367845376 7669 33554432000 4194304 5683644 140733698290352 140733698285184 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/5388/statm: 89806 7669 1902 364 0 25029 0
[pid=5395] ppid=5388 vsize=283444 CPUtime=0 cores=0,2,4,6
/proc/5395/stat : 5395 (python) S 5388 5388 2614 0 -1 4202560 702 0 0 0 0 0 0 0 20 0 1 0 337899167 290246656 6068 33554432000 4194304 5683644 140733698290352 140733698281440 240565191080 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/5395/statm: 70861 6068 307 364 0 6084 0
[pid=5397] ppid=5395 vsize=4612 CPUtime=2.37 cores=0,2,4,6
/proc/5397/stat : 5397 (clasp-1.3.7-x86) R 5395 5397 5397 0 -1 4202496 1177 0 0 0 237 0 0 0 20 0 1 0 337899167 4722688 909 33554432000 134512640 136139019 4290128384 4290127104 134875024 0 0 16781312 16386 0 0 0 17 0 0 0 0 0 0
/proc/5397/statm: 1153 909 182 398 0 753 0
[pid=5388/tid=5396] ppid=5386 vsize=359224 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5396/stat : 5396 (python) S 5386 5388 2614 0 -1 4202560 27 1417 0 0 0 0 3 0 20 0 2 0 337899167 367845376 7669 33554432000 4194304 5683644 140733698290352 140436765745712 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 2.99
Current children cumulated vsize (KiB) 647280

[startup+6.30034 s]
/proc/loadavg: 2.16 2.08 2.09 3/222 5397
/proc/meminfo: memFree=28615088/32873844 swapFree=7116/7116
[pid=5388] ppid=5386 vsize=359224 CPUtime=0.62 cores=0,2,4,6
/proc/5388/stat : 5388 (python) S 5386 5388 2614 0 -1 4202496 11923 1417 1 0 48 11 3 0 20 0 2 0 337899095 367845376 7669 33554432000 4194304 5683644 140733698290352 140733698285184 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/5388/statm: 89806 7669 1902 364 0 25029 0
[pid=5395] ppid=5388 vsize=283444 CPUtime=0 cores=0,2,4,6
/proc/5395/stat : 5395 (python) S 5388 5388 2614 0 -1 4202560 845 0 0 0 0 0 0 0 20 0 1 0 337899167 290246656 6090 33554432000 4194304 5683644 140733698290352 140733698281440 240565191080 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/5395/statm: 70861 6090 329 364 0 6084 0
[pid=5397] ppid=5395 vsize=5404 CPUtime=5.57 cores=0,2,4,6
/proc/5397/stat : 5397 (clasp-1.3.7-x86) R 5395 5397 5397 0 -1 4202496 1369 0 0 0 557 0 0 0 20 0 1 0 337899167 5533696 1101 33554432000 134512640 136139019 4290128384 4290127248 134831284 0 0 16781312 16386 0 0 0 17 2 0 0 0 0 0
/proc/5397/statm: 1351 1101 182 398 0 951 0
[pid=5388/tid=5396] ppid=5386 vsize=359224 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5396/stat : 5396 (python) S 5386 5388 2614 0 -1 4202560 27 1417 0 0 0 0 3 0 20 0 2 0 337899167 367845376 7669 33554432000 4194304 5683644 140733698290352 140436765745712 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 6.19
Current children cumulated vsize (KiB) 648072

[startup+12.7069 s]
/proc/loadavg: 2.15 2.08 2.09 3/222 5397
/proc/meminfo: memFree=28613988/32873844 swapFree=7116/7116
[pid=5388] ppid=5386 vsize=359224 CPUtime=0.62 cores=0,2,4,6
/proc/5388/stat : 5388 (python) S 5386 5388 2614 0 -1 4202496 11923 1417 1 0 48 11 3 0 20 0 2 0 337899095 367845376 7669 33554432000 4194304 5683644 140733698290352 140733698285184 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/5388/statm: 89806 7669 1902 364 0 25029 0
[pid=5395] ppid=5388 vsize=283444 CPUtime=0.01 cores=0,2,4,6
/proc/5395/stat : 5395 (python) S 5388 5388 2614 0 -1 4202560 853 0 0 0 1 0 0 0 20 0 1 0 337899167 290246656 6091 33554432000 4194304 5683644 140733698290352 140733698281440 240565191080 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/5395/statm: 70861 6091 330 364 0 6084 0
[pid=5397] ppid=5395 vsize=5932 CPUtime=11.96 cores=0,2,4,6
/proc/5397/stat : 5397 (clasp-1.3.7-x86) R 5395 5397 5397 0 -1 4202496 1485 0 0 0 1196 0 0 0 20 0 1 0 337899167 6074368 1217 33554432000 134512640 136139019 4290128384 4290127296 134640576 0 0 16781312 16386 0 0 0 17 0 0 0 0 0 0
/proc/5397/statm: 1483 1217 182 398 0 1083 0
[pid=5388/tid=5396] ppid=5386 vsize=359224 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5396/stat : 5396 (python) S 5386 5388 2614 0 -1 4202560 27 1417 0 0 0 0 3 0 20 0 2 0 337899167 367845376 7669 33554432000 4194304 5683644 140733698290352 140436765745712 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 12.59
Current children cumulated vsize (KiB) 648600

[startup+25.5003 s]
/proc/loadavg: 2.13 2.07 2.09 3/222 5397
/proc/meminfo: memFree=28613732/32873844 swapFree=7116/7116
[pid=5388] ppid=5386 vsize=359224 CPUtime=0.62 cores=0,2,4,6
/proc/5388/stat : 5388 (python) S 5386 5388 2614 0 -1 4202496 11923 1417 1 0 48 11 3 0 20 0 2 0 337899095 367845376 7669 33554432000 4194304 5683644 140733698290352 140733698285184 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/5388/statm: 89806 7669 1902 364 0 25029 0
[pid=5395] ppid=5388 vsize=283444 CPUtime=0.03 cores=0,2,4,6
/proc/5395/stat : 5395 (python) S 5388 5388 2614 0 -1 4202560 861 0 0 0 2 1 0 0 20 0 1 0 337899167 290246656 6091 33554432000 4194304 5683644 140733698290352 140733698281440 240565191080 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/5395/statm: 70861 6091 330 364 0 6084 0
[pid=5397] ppid=5395 vsize=6548 CPUtime=24.75 cores=0,2,4,6
/proc/5397/stat : 5397 (clasp-1.3.7-x86) R 5395 5397 5397 0 -1 4202496 1644 0 0 0 2475 0 0 0 20 0 1 0 337899167 6705152 1376 33554432000 134512640 136139019 4290128384 4290127248 134831224 0 0 16781312 16386 0 0 0 17 0 0 0 0 0 0
/proc/5397/statm: 1637 1376 182 398 0 1237 0
[pid=5388/tid=5396] ppid=5386 vsize=359224 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5396/stat : 5396 (python) S 5386 5388 2614 0 -1 4202560 27 1417 0 0 0 0 3 0 20 0 2 0 337899167 367845376 7669 33554432000 4194304 5683644 140733698290352 140436765745712 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 25.4
Current children cumulated vsize (KiB) 649216

[startup+51.1004 s]
/proc/loadavg: 2.08 2.06 2.08 3/226 5401
/proc/meminfo: memFree=28573076/32873844 swapFree=7116/7116
[pid=5388] ppid=5386 vsize=435016 CPUtime=0.65 cores=0,2,4,6
/proc/5388/stat : 5388 (python) S 5386 5388 2614 0 -1 4202496 12886 1417 1 0 51 11 3 0 20 0 3 0 337899095 445456384 7678 33554432000 4194304 5683644 140733698290352 140733698285184 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/5388/statm: 108754 7678 1905 364 0 43974 0
[pid=5395] ppid=5388 vsize=359224 CPUtime=0.07 cores=0,2,4,6
/proc/5395/stat : 5395 (python) S 5388 5388 2614 0 -1 4202560 1148 0 0 0 4 3 0 0 20 0 2 0 337899167 367845376 6164 33554432000 4194304 5683644 140733698290352 140733698282000 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/5395/statm: 89806 6164 396 364 0 25029 0
[pid=5397] ppid=5395 vsize=7524 CPUtime=47.1 cores=0,2,4,6
/proc/5397/stat : 5397 (clasp-1.3.7-x86) T 5395 5397 5397 0 -1 4202496 1895 0 0 0 4710 0 0 0 20 0 1 0 337899167 7704576 1627 33554432000 134512640 136139019 4290128384 4290127360 134883859 0 0 16781312 16386 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/5397/statm: 1881 1627 182 398 0 1481 0
[pid=5395/tid=5398] ppid=5388 vsize=359224 CPUtime=0 cores=0,2,4,6
/proc/5395/task/5398/stat : 5398 (python) S 5388 5388 2614 0 -1 4202560 44 0 0 0 0 0 0 0 20 0 2 0 337903880 367845376 6164 33554432000 4194304 5683644 140733698290352 140436765745712 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=5399] ppid=5388 vsize=359236 CPUtime=0 cores=0,2,4,6
/proc/5399/stat : 5399 (python) S 5388 5388 2614 0 -1 4202560 753 0 0 0 0 0 0 0 20 0 1 0 337903883 367857664 6075 33554432000 4194304 5683644 140733698290352 140733698281440 240565191139 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/5399/statm: 89809 6075 308 364 0 25029 0
[pid=5401] ppid=5399 vsize=48988 CPUtime=3.21 cores=0,2,4,6
/proc/5401/stat : 5401 (scip-2.0.1.linu) R 5399 5401 5401 0 -1 4202496 10805 0 0 0 313 8 0 0 20 0 1 0 337903883 50163712 10029 33554432000 4194304 13684479 140734465868080 140734465859992 11876362 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/5401/statm: 12247 10029 764 2317 0 9925 0
[pid=5388/tid=5396] ppid=5386 vsize=435016 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5396/stat : 5396 (python) S 5386 5388 2614 0 -1 4202560 27 1417 0 0 0 0 3 0 20 0 3 0 337899167 445456384 7678 33554432000 4194304 5683644 140733698290352 140436765745712 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=5388/tid=5400] ppid=5386 vsize=435016 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5400/stat : 5400 (python) S 5386 5388 2614 0 -1 4202560 21 1417 0 0 0 0 3 0 20 0 3 0 337903883 445456384 7678 33554432000 4194304 5683644 140733698290352 140436755255856 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 51.03
Current children cumulated vsize (KiB) 1209988

[startup+102.301 s]
/proc/loadavg: 2.03 2.05 2.08 3/230 5405
/proc/meminfo: memFree=28468412/32873844 swapFree=7116/7116
[pid=5388] ppid=5386 vsize=510808 CPUtime=0.67 cores=0,2,4,6
/proc/5388/stat : 5388 (python) S 5386 5388 2614 0 -1 4202496 13835 1417 1 0 53 11 3 0 20 0 4 0 337899095 523067392 7686 33554432000 4194304 5683644 140733698290352 140733698285184 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/5388/statm: 127702 7686 1907 364 0 62919 0
[pid=5395] ppid=5388 vsize=359224 CPUtime=0.07 cores=0,2,4,6
/proc/5395/stat : 5395 (python) S 5388 5388 2614 0 -1 4202560 1148 0 0 0 4 3 0 0 20 0 2 0 337899167 367845376 6164 33554432000 4194304 5683644 140733698290352 140733698282000 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/5395/statm: 89806 6164 396 364 0 25029 0
[pid=5397] ppid=5395 vsize=7524 CPUtime=47.1 cores=0,2,4,6
/proc/5397/stat : 5397 (clasp-1.3.7-x86) T 5395 5397 5397 0 -1 4202496 1895 0 0 0 4710 0 0 0 20 0 1 0 337899167 7704576 1627 33554432000 134512640 136139019 4290128384 4290127360 134883859 0 0 16781312 16386 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/5397/statm: 1881 1627 182 398 0 1481 0
[pid=5395/tid=5398] ppid=5388 vsize=359224 CPUtime=0 cores=0,2,4,6
/proc/5395/task/5398/stat : 5398 (python) S 5388 5388 2614 0 -1 4202560 44 0 0 0 0 0 0 0 20 0 2 0 337903880 367845376 6164 33554432000 4194304 5683644 140733698290352 140436765745712 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=5399] ppid=5388 vsize=359236 CPUtime=0.07 cores=0,2,4,6
/proc/5399/stat : 5399 (python) S 5388 5388 2614 0 -1 4202560 1093 0 0 0 4 3 0 0 20 0 2 0 337903883 367857664 6147 33554432000 4194304 5683644 140733698290352 140733698282000 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/5399/statm: 89809 6147 379 364 0 25029 0
[pid=5401] ppid=5399 vsize=84648 CPUtime=48.05 cores=0,2,4,6
/proc/5401/stat : 5401 (scip-2.0.1.linu) T 5399 5401 5401 0 -1 4202496 24423 0 0 0 4762 43 0 0 20 0 1 0 337903883 86679552 18802 33554432000 4194304 13684479 140734465868080 140734465862008 7354541 0 0 16781312 2 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/5401/statm: 21162 18802 847 2317 0 18840 0
[pid=5399/tid=5402] ppid=5388 vsize=359236 CPUtime=0 cores=0,2,4,6
/proc/5399/task/5402/stat : 5402 (python) S 5388 5388 2614 0 -1 4202560 49 0 0 0 0 0 0 0 20 0 2 0 337908692 367857664 6147 33554432000 4194304 5683644 140733698290352 140436765745712 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=5403] ppid=5388 vsize=435028 CPUtime=0 cores=0,2,4,6
/proc/5403/stat : 5403 (python) S 5388 5388 2614 0 -1 4202560 892 0 0 0 0 0 0 0 20 0 1 0 337908694 445468672 6104 33554432000 4194304 5683644 140733698290352 140733698281440 240565191139 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0

################
# More data... #
################

[pid=5388/tid=5400] ppid=5386 vsize=966064 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5400/stat : 5400 (python) S 5386 5388 2614 0 -1 4202560 36 1417 0 0 0 0 3 0 20 0 10 0 337903883 989249536 8368 33554432000 4194304 5683644 140733698290352 140436755255856 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=5388/tid=5404] ppid=5386 vsize=966064 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5404/stat : 5404 (python) S 5386 5388 2614 0 -1 4202560 23 1417 0 0 0 0 3 0 20 0 10 0 337908694 989249536 8368 33554432000 4194304 5683644 140733698290352 140436744766000 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=5388/tid=5411] ppid=5386 vsize=966064 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5411/stat : 5411 (python) S 5386 5388 2614 0 -1 4202560 22 1417 0 0 0 0 3 0 20 0 10 0 337917889 989249536 8368 33554432000 4194304 5683644 140733698290352 140436734276144 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=5388/tid=5417] ppid=5386 vsize=966064 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5417/stat : 5417 (python) S 5386 5388 2614 0 -1 4202560 21 1417 0 0 0 0 3 0 20 0 10 0 337927345 989249536 8368 33554432000 4194304 5683644 140733698290352 140436723786288 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=5388/tid=5425] ppid=5386 vsize=966064 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5425/stat : 5425 (python) S 5386 5388 2614 0 -1 4202560 20 1417 0 0 0 0 3 0 20 0 10 0 337950627 989249536 8368 33554432000 4194304 5683644 140733698290352 140436370874928 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=5388/tid=5505] ppid=5386 vsize=966064 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5505/stat : 5505 (python) S 5386 5388 2614 0 -1 4202560 23 1417 0 0 0 0 3 0 20 0 10 0 337996365 989249536 8368 33554432000 4194304 5683644 140733698290352 140436360385072 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=5388/tid=5513] ppid=5386 vsize=966064 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5513/stat : 5513 (python) S 5386 5388 2614 0 -1 4202560 21 1417 0 0 0 0 3 0 20 0 10 0 338001173 989249536 8368 33554432000 4194304 5683644 140733698290352 140436349895216 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=5388/tid=5538] ppid=5386 vsize=966064 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5538/stat : 5538 (python) S 5386 5388 2614 0 -1 4202560 21 1417 0 0 0 0 3 0 20 0 10 0 338065587 989249536 8368 33554432000 4194304 5683644 140733698290352 140436339405360 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1670.57
Current children cumulated vsize (KiB) 16610932

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1665.9 s]
/proc/loadavg: 2.01 2.02 2.02 3/255 5539
/proc/meminfo: memFree=28181768/32873844 swapFree=7116/7116
[pid=5388] ppid=5386 vsize=966064 CPUtime=1.32 cores=0,2,4,6
/proc/5388/stat : 5388 (python) S 5386 5388 2614 0 -1 4202496 20002 1417 1 0 116 13 3 0 20 0 10 0 337899095 989249536 8368 33554432000 4194304 5683644 140733698290352 140733698285184 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/5388/statm: 241516 8368 1919 364 0 176715 0
[pid=5395] ppid=5388 vsize=359224 CPUtime=0.39 cores=0,2,4,6
/proc/5395/stat : 5395 (python) S 5388 5388 2614 0 -1 4202560 1222 0 0 0 22 17 0 0 20 0 2 0 337899167 367845376 6165 33554432000 4194304 5683644 140733698290352 140733698282000 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/5395/statm: 89806 6165 397 364 0 25029 0
[pid=5397] ppid=5395 vsize=9340 CPUtime=229.54 cores=0,2,4,6
/proc/5397/stat : 5397 (clasp-1.3.7-x86) T 5395 5397 5397 0 -1 4202496 2386 0 0 0 22954 0 0 0 20 0 1 0 337899167 9564160 2083 33554432000 134512640 136139019 4290128384 4290127360 134883437 0 0 16781312 16386 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/5397/statm: 2335 2083 182 398 0 1935 0
[pid=5395/tid=5398] ppid=5388 vsize=359224 CPUtime=0 cores=0,2,4,6
/proc/5395/task/5398/stat : 5398 (python) S 5388 5388 2614 0 -1 4202560 44 0 0 0 0 0 0 0 20 0 2 0 337903880 367845376 6165 33554432000 4194304 5683644 140733698290352 140436765745712 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=5399] ppid=5388 vsize=359236 CPUtime=1.78 cores=0,2,4,6
/proc/5399/stat : 5399 (python) S 5388 5388 2614 0 -1 4202560 1255 0 0 0 106 72 0 0 20 0 2 0 337903883 367857664 6147 33554432000 4194304 5683644 140733698290352 140733698282000 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/5399/statm: 89809 6147 379 364 0 25029 0
[pid=5401] ppid=5399 vsize=120816 CPUtime=1061.79 cores=0,2,4,6
/proc/5401/stat : 5401 (scip-2.0.1.linu) T 5399 5401 5401 0 -1 4202496 33784 0 0 0 105297 882 0 0 20 0 1 0 337903883 123715584 28413 33554432000 4194304 13684479 140734465868080 140734465862232 7268004 0 0 16781312 2 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/5401/statm: 30204 28413 847 2317 0 27882 0
[pid=5399/tid=5402] ppid=5388 vsize=359236 CPUtime=0 cores=0,2,4,6
/proc/5399/task/5402/stat : 5402 (python) S 5388 5388 2614 0 -1 4202560 49 0 0 0 0 0 0 0 20 0 2 0 337908692 367857664 6147 33554432000 4194304 5683644 140733698290352 140436765745712 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=5403] ppid=5388 vsize=435028 CPUtime=0.15 cores=0,2,4,6
/proc/5403/stat : 5403 (python) S 5388 5388 2614 0 -1 4202560 1148 0 0 0 8 7 0 0 20 0 2 0 337908694 445468672 6153 33554432000 4194304 5683644 140733698290352 140733698282000 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/5403/statm: 108757 6153 379 364 0 43974 0
[pid=5405] ppid=5403 vsize=78640 CPUtime=93.57 cores=0,2,4,6
/proc/5405/stat : 5405 (scip-2.0.1.linu) T 5403 5405 5405 0 -1 4202496 28230 0 0 0 9309 48 0 0 20 0 1 0 337908694 80527360 17454 33554432000 4194304 15243407 140733740388672 140733740384472 8113833 0 0 16781312 2 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/5405/statm: 19660 17454 967 2698 0 16955 0
[pid=5403/tid=5406] ppid=5388 vsize=435028 CPUtime=0 cores=0,2,4,6
/proc/5403/task/5406/stat : 5406 (python) S 5388 5388 2614 0 -1 4202560 46 0 0 0 0 0 0 0 20 0 2 0 337913332 445468672 6153 33554432000 4194304 5683644 140733698290352 140436755255856 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=5410] ppid=5388 vsize=510820 CPUtime=0.08 cores=0,2,4,6
/proc/5410/stat : 5410 (python) S 5388 5388 2614 0 -1 4202560 1123 0 0 0 5 3 0 0 20 0 2 0 337917889 523079680 6159 33554432000 4194304 5683644 140733698290352 140733698282000 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/5410/statm: 127705 6159 379 364 0 62919 0
[pid=5412] ppid=5410 vsize=74856 CPUtime=47.26 cores=0,2,4,6
/proc/5412/stat : 5412 (minion) T 5410 5412 5412 0 -1 4202496 4845 0 0 0 4725 1 0 0 20 0 1 0 337917889 76652544 11463 33554432000 134512640 165565024 4292526704 4292523936 159737071 0 0 16781312 8396802 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/5412/statm: 18714 11463 358 7582 0 11129 0
[pid=5410/tid=5414] ppid=5388 vsize=510820 CPUtime=0 cores=0,2,4,6
/proc/5410/task/5414/stat : 5414 (python) S 5388 5388 2614 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 2 0 337922616 523079680 6159 33554432000 4194304 5683644 140733698290352 140436744766000 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=5416] ppid=5388 vsize=586612 CPUtime=0.08 cores=0,2,4,6
/proc/5416/stat : 5416 (python) S 5388 5388 2614 0 -1 4202560 1168 0 0 0 4 4 0 0 20 0 2 0 337927345 600690688 6165 33554432000 4194304 5683644 140733698290352 140733698282000 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/5416/statm: 146653 6165 379 364 0 81864 0
[pid=5418] ppid=5416 vsize=113168 CPUtime=47.14 cores=0,2,4,6
/proc/5418/stat : 5418 (pbct-0.1.2-linu) T 5416 5418 5418 0 -1 4202496 28813 0 0 0 4652 62 0 0 20 0 1 0 337927345 115884032 27326 33554432000 134512640 136422188 4292536896 4292534240 135026560 0 0 16781312 16387 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/5418/statm: 28292 27326 214 467 0 27823 0
[pid=5416/tid=5419] ppid=5388 vsize=586612 CPUtime=0 cores=0,2,4,6
/proc/5416/task/5419/stat : 5419 (python) S 5388 5388 2614 0 -1 4202560 47 0 0 0 0 0 0 0 20 0 2 0 337932062 600690688 6165 33554432000 4194304 5683644 140733698290352 140436734276144 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=5424] ppid=5388 vsize=662908 CPUtime=0.08 cores=0,2,4,6
/proc/5424/stat : 5424 (python) S 5388 5388 2614 0 -1 4202560 1135 0 0 0 4 4 0 0 20 0 2 0 337950627 678817792 6297 33554432000 4194304 5683644 140733698290352 140733698282000 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/5424/statm: 165727 6297 379 364 0 100935 0
[pid=5426] ppid=5424 vsize=9624972 CPUtime=45.97 cores=0,2,4,6
/proc/5426/stat : 5426 (java) T 5424 5426 5426 0 -1 4202496 24171 0 1 0 4568 29 0 0 20 0 19 0 337950627 9855971328 263991 33554432000 1073741824 1073778376 140734818404032 140734818395168 240568533053 0 0 0 16800975 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/5426/statm: 2406243 263991 2271 9 0 2396937 0
[pid=5426/tid=5427] ppid=5424 vsize=9624972 CPUtime=43.14 cores=0,2,4,6
/proc/5426/task/5427/stat : 5427 (java) T 5424 5426 5426 0 -1 4202560 3344 0 1 0 4298 16 0 0 20 0 19 0 337950629 9855971328 263991 33554432000 1073741824 1073778376 140734818404032 139912892005472 139912737935709 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=5426/tid=5428] ppid=5424 vsize=9624972 CPUtime=0.08 cores=0,2,4,6
/proc/5426/task/5428/stat : 5428 (java) T 5424 5426 5426 0 -1 4202560 24 0 0 0 8 0 0 0 20 0 19 0 337950630 9855971328 263991 33554432000 1073741824 1073778376 140734818404032 139912864459560 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=5426/tid=5429] ppid=5424 vsize=9624972 CPUtime=0.1 cores=0,2,4,6
/proc/5426/task/5429/stat : 5429 (java) T 5424 5426 5426 0 -1 4202560 76 0 0 0 10 0 0 0 20 0 19 0 337950630 9855971328 263991 33554432000 1073741824 1073778376 140734818404032 139912863406760 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=5426/tid=5430] ppid=5424 vsize=9624972 CPUtime=0.06 cores=0,2,4,6
/proc/5426/task/5430/stat : 5430 (java) T 5424 5426 5426 0 -1 4202560 44 0 0 0 5 1 0 0 20 0 19 0 337950630 9855971328 263991 33554432000 1073741824 1073778376 140734818404032 139912862354472 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=5426/tid=5431] ppid=5424 vsize=9624972 CPUtime=0.1 cores=0,2,4,6
/proc/5426/task/5431/stat : 5431 (java) T 5424 5426 5426 0 -1 4202560 55 0 0 0 8 2 0 0 20 0 19 0 337950630 9855971328 263991 33554432000 1073741824 1073778376 140734818404032 139912861301672 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=5426/tid=5432] ppid=5424 vsize=9624972 CPUtime=0.08 cores=0,2,4,6
/proc/5426/task/5432/stat : 5432 (java) T 5424 5426 5426 0 -1 4202560 26 0 0 0 8 0 0 0 20 0 19 0 337950630 9855971328 263991 33554432000 1073741824 1073778376 140734818404032 139912860248360 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=5426/tid=5433] ppid=5424 vsize=9624972 CPUtime=0.08 cores=0,2,4,6
/proc/5426/task/5433/stat : 5433 (java) T 5424 5426 5426 0 -1 4202560 25 0 0 0 8 0 0 0 20 0 19 0 337950630 9855971328 263991 33554432000 1073741824 1073778376 140734818404032 139912859195560 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=5426/tid=5434] ppid=5424 vsize=9624972 CPUtime=0.11 cores=0,2,4,6
/proc/5426/task/5434/stat : 5434 (java) T 5424 5426 5426 0 -1 4202560 58 0 0 0 9 2 0 0 20 0 19 0 337950630 9855971328 263991 33554432000 1073741824 1073778376 140734818404032 139912858143272 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=5426/tid=5435] ppid=5424 vsize=9624972 CPUtime=0.09 cores=0,2,4,6
/proc/5426/task/5435/stat : 5435 (java) T 5424 5426 5426 0 -1 4202560 37 0 0 0 7 2 0 0 20 0 19 0 337950630 9855971328 263991 33554432000 1073741824 1073778376 140734818404032 139912857090472 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=5426/tid=5436] ppid=5424 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/5426/task/5436/stat : 5436 (java) T 5424 5426 5426 0 -1 4202560 439 0 0 0 0 0 0 0 20 0 19 0 337950631 9855971328 263991 33554432000 1073741824 1073778376 140734818404032 139912245385960 240568547241 0 0 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=5426/tid=5437] ppid=5424 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/5426/task/5437/stat : 5437 (java) T 5424 5426 5426 0 -1 4202560 17 0 0 0 0 0 0 0 20 0 19 0 337950632 9855971328 263991 33554432000 1073741824 1073778376 140734818404032 139912244332024 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=5426/tid=5438] ppid=5424 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/5426/task/5438/stat : 5438 (java) T 5424 5426 5426 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 19 0 337950632 9855971328 263991 33554432000 1073741824 1073778376 140734818404032 139912243279624 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=5426/tid=5439] ppid=5424 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/5426/task/5439/stat : 5439 (java) T 5424 5426 5426 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 337950634 9855971328 263991 33554432000 1073741824 1073778376 140734818404032 139912242228112 240568555296 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=5426/tid=5440] ppid=5424 vsize=9624972 CPUtime=1.02 cores=0,2,4,6
/proc/5426/task/5440/stat : 5440 (java) T 5424 5426 5426 0 -1 4202560 9401 0 0 0 100 2 0 0 20 0 19 0 337950634 9855971328 263991 33554432000 1073741824 1073778376 140734818404032 139912241174712 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=5426/tid=5441] ppid=5424 vsize=9624972 CPUtime=0.83 cores=0,2,4,6
/proc/5426/task/5441/stat : 5441 (java) T 5424 5426 5426 0 -1 4202560 9225 0 0 0 81 2 0 0 20 0 19 0 337950634 9855971328 263991 33554432000 1073741824 1073778376 140734818404032 139912240121912 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=5426/tid=5442] ppid=5424 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/5426/task/5442/stat : 5442 (java) T 5424 5426 5426 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 337950634 9855971328 263991 33554432000 1073741824 1073778376 140734818404032 139912239069816 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=5426/tid=5443] ppid=5424 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/5426/task/5443/stat : 5443 (java) T 5424 5426 5426 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 19 0 337950634 9855971328 263991 33554432000 1073741824 1073778376 140734818404032 139912238017224 240568547241 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=5426/tid=5444] ppid=5424 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/5426/task/5444/stat : 5444 (java) T 5424 5426 5426 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 19 0 337950670 9855971328 263991 33554432000 1073741824 1073778376 140734818404032 139912236963240 240568547241 0 4 0 16800975 18446744071579387992 0 0 -1 4 0 0 0 0 0
[pid=5424/tid=5445] ppid=5388 vsize=662908 CPUtime=0 cores=0,2,4,6
/proc/5424/task/5445/stat : 5445 (python) S 5388 5388 2614 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 337954982 678817792 6297 33554432000 4194304 5683644 140733698290352 140436723786288 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=5504] ppid=5388 vsize=738700 CPUtime=0.07 cores=0,2,4,6
/proc/5504/stat : 5504 (python) S 5388 5388 2614 0 -1 4202560 1156 0 0 0 5 2 0 0 20 0 2 0 337996365 756428800 6810 33554432000 4194304 5683644 140733698290352 140733698282000 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/5504/statm: 184675 6810 379 364 0 119880 0
[pid=5506] ppid=5504 vsize=77992 CPUtime=48.05 cores=0,2,4,6
/proc/5506/stat : 5506 (bsolo_pb10) T 5504 5506 5506 0 -1 4202496 17465 0 0 0 4798 7 0 0 20 0 1 0 337996365 79863808 17074 33554432000 134512640 137138936 4294109152 4294107864 136504096 0 0 16781312 16384 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/5506/statm: 19498 17074 192 642 0 18854 0
[pid=5504/tid=5511] ppid=5388 vsize=738700 CPUtime=0 cores=0,2,4,6
/proc/5504/task/5511/stat : 5511 (python) S 5388 5388 2614 0 -1 4202560 43 0 0 0 0 0 0 0 20 0 2 0 338001172 756428800 6810 33554432000 4194304 5683644 140733698290352 140436370874928 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=5512] ppid=5388 vsize=814492 CPUtime=0.16 cores=0,2,4,6
/proc/5512/stat : 5512 (python) S 5388 5388 2614 0 -1 4202560 1236 0 0 0 9 7 0 0 20 0 2 0 338001173 834039808 6823 33554432000 4194304 5683644 140733698290352 140733698282000 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/5512/statm: 203623 6823 385 364 0 138825 0
[pid=5514] ppid=5512 vsize=178036 CPUtime=92.16 cores=0,2,4,6
/proc/5514/stat : 5514 (bsolo_pb10) T 5512 5514 5514 0 -1 4202496 42465 0 0 0 9187 29 0 0 20 0 1 0 338001174 182308864 42076 33554432000 134512640 137138936 4286642528 4286641252 136501693 0 0 16781312 16384 18446744071579387992 0 0 17 0 0 0 0 0 0
/proc/5514/statm: 44509 42076 190 642 0 43865 0
[pid=5512/tid=5523] ppid=5388 vsize=814492 CPUtime=0 cores=0,2,4,6
/proc/5512/task/5523/stat : 5523 (python) S 5388 5388 2614 0 -1 4202560 44 0 0 0 0 0 0 0 20 0 2 0 338010392 834039808 6823 33554432000 4194304 5683644 140733698290352 140436360385072 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=5537] ppid=5388 vsize=890284 CPUtime=0 cores=0,2,4,6
/proc/5537/stat : 5537 (python) S 5388 5388 2614 0 -1 4202560 864 0 0 0 0 0 0 0 20 0 1 0 338065586 911650816 6758 33554432000 4194304 5683644 140733698290352 140733698281440 240565191139 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/5537/statm: 222571 6758 315 364 0 157770 0
[pid=5539] ppid=5537 vsize=9744 CPUtime=0.98 cores=0,2,4,6
/proc/5539/stat : 5539 (wbo1.4a) R 5537 5539 5539 0 -1 4202496 2325 0 0 0 98 0 0 0 20 0 1 0 338065587 9977856 1998 33554432000 134512640 135751879 4294284336 4294282588 135270609 0 0 16781312 3 0 0 0 17 4 0 0 0 0 0
/proc/5539/statm: 2436 1998 143 303 0 2131 0
[pid=5388/tid=5396] ppid=5386 vsize=966064 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5396/stat : 5396 (python) S 5386 5388 2614 0 -1 4202560 36 1417 0 0 0 0 3 0 20 0 10 0 337899167 989249536 8368 33554432000 4194304 5683644 140733698290352 140436765745712 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=5388/tid=5400] ppid=5386 vsize=966064 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5400/stat : 5400 (python) S 5386 5388 2614 0 -1 4202560 36 1417 0 0 0 0 3 0 20 0 10 0 337903883 989249536 8368 33554432000 4194304 5683644 140733698290352 140436755255856 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=5388/tid=5404] ppid=5386 vsize=966064 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5404/stat : 5404 (python) S 5386 5388 2614 0 -1 4202560 23 1417 0 0 0 0 3 0 20 0 10 0 337908694 989249536 8368 33554432000 4194304 5683644 140733698290352 140436744766000 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=5388/tid=5411] ppid=5386 vsize=966064 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5411/stat : 5411 (python) S 5386 5388 2614 0 -1 4202560 22 1417 0 0 0 0 3 0 20 0 10 0 337917889 989249536 8368 33554432000 4194304 5683644 140733698290352 140436734276144 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=5388/tid=5417] ppid=5386 vsize=966064 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5417/stat : 5417 (python) S 5386 5388 2614 0 -1 4202560 21 1417 0 0 0 0 3 0 20 0 10 0 337927345 989249536 8368 33554432000 4194304 5683644 140733698290352 140436723786288 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=5388/tid=5425] ppid=5386 vsize=966064 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5425/stat : 5425 (python) S 5386 5388 2614 0 -1 4202560 20 1417 0 0 0 0 3 0 20 0 10 0 337950627 989249536 8368 33554432000 4194304 5683644 140733698290352 140436370874928 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=5388/tid=5505] ppid=5386 vsize=966064 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5505/stat : 5505 (python) S 5386 5388 2614 0 -1 4202560 23 1417 0 0 0 0 3 0 20 0 10 0 337996365 989249536 8368 33554432000 4194304 5683644 140733698290352 140436360385072 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=5388/tid=5513] ppid=5386 vsize=966064 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5513/stat : 5513 (python) S 5386 5388 2614 0 -1 4202560 21 1417 0 0 0 0 3 0 20 0 10 0 338001173 989249536 8368 33554432000 4194304 5683644 140733698290352 140436349895216 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=5388/tid=5538] ppid=5386 vsize=966064 CPUtime=0.03 cores=0,2,4,6
/proc/5388/task/5538/stat : 5538 (python) S 5386 5388 2614 0 -1 4202560 21 1417 0 0 0 0 3 0 20 0 10 0 338065587 989249536 8368 33554432000 4194304 5683644 140733698290352 140436339405360 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1670.57
Current children cumulated vsize (KiB) 16610932

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 5388 and gives
#  childrusage.ru_utime.tv_sec=1
#  childrusage.ru_utime.tv_usec=207816
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=140978
# CPU time returned by wait4() is 1.34879
# while last known CPU time is 1670.57
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1665.97
CPU time (s): 1670.57
CPU user time (s): 1658.7
CPU system time (s): 11.87
CPU usage (%): 100.276
Max. virtual memory (cumulated for all children) (KiB): 16610932

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1.20782
system time used= 0.140978
maximum resident set size= 33472
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 21906
page faults= 1
swaps= 0
block input operations= 0
block output operations= 1728
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3460
involuntary context switches= 30

runsolver used 8.93064 second user time and 21.0428 second system time

The end

Launcher Data 

Begin job on node146 at 2012-06-03 12:02:21
IDJOB=3738043
IDBENCH=48376
IDSOLVER=2327
FILE ID=node146/3738043-1338717741
RUNJOBID= node146-1338713700-2630
PBS_JOBID= 14636739
Free space on /tmp= 70808 MiB

SOLVER NAME= PB11: borg pb-dec-11.04.03
BENCH NAME= PB07/normalized-PB07/SATUNSAT-SMALLINT-NLC/submittedPB07/manquinho/dbsg/normalized-dbsg_100_25_2_15.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.dec.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3738043-1338717741/watcher-3738043-1338717741 -o /tmp/evaluation-result-3738043-1338717741/solver-3738043-1338717741 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3738043-1338717741.opb -seed 1276299188 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= b437bd5db918ab114396d43dfca73285
RANDOM SEED=1276299188

node146.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        28644408 kB
Buffers:          214156 kB
Cached:           596732 kB
SwapCached:         1792 kB
Active:          3149440 kB
Inactive:         286272 kB
Active(anon):    2625744 kB
Inactive(anon):     1764 kB
Active(file):     523696 kB
Inactive(file):   284508 kB
Unevictable:        7116 kB
Mlocked:            7116 kB
SwapTotal:      67108856 kB
SwapFree:       67099888 kB
Dirty:            101884 kB
Writeback:            52 kB
AnonPages:       2631156 kB
Mapped:            32948 kB
Shmem:                84 kB
Slab:             643140 kB
SReclaimable:      81240 kB
SUnreclaim:       561900 kB
KernelStack:        1768 kB
PageTables:        13628 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:    3435228 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:   2242560 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70700 MiB
End job on node146 at 2012-06-03 12:30:10