Trace number 3347309

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 160003 5006.84

General information on the benchmark

NameAPPLICATIONS/satComp09_BioInstances/
rpoc_xits_09_UNSAT.cnf
MD5SUMfeb620c17f2a4d7899a2e7883baa7391
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark140865
Satisfiable
(Un)Satisfiability was proved
Number of variables1430
Number of clauses87044
Sum of the clauses size258924
Maximum clause length10
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 22369
Number of clauses of size 384652
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 523

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 32 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3347309-1304998713/watcher-3347309-1304998713 -o /tmp/evaluation-result-3347309-1304998713/solver-3347309-1304998713 -C 160000 -W 5100 -M 266338304 ppfolio HOME/instance-3347309-1304998713.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 272730423296 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 272730474496 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 27.21 31.05 31.73 2/413 31912
/proc/meminfo: memFree=256506728/264257624 swapFree=67111528/67111528
[pid=31912] ppid=31910 vsize=18324 CPUtime=0 cores=0-31
/proc/31912/stat : 31912 (ppfolio) S 31910 31912 31886 0 -1 4202496 513 0 0 0 0 0 0 0 24 0 1 0 611357710 18763776 385 268435456000 4194304 4287625 140734349320608 18446744073709551615 258650975999 0 0 4096 65536 9184107106532327423 0 0 17 4 0 0 0
/proc/31912/statm: 4581 385 337 23 0 91 0

[startup+0.0524129 s]
/proc/loadavg: 27.21 31.05 31.73 2/413 31912
/proc/meminfo: memFree=256506728/264257624 swapFree=67111528/67111528
[pid=31912] ppid=31910 vsize=18324 CPUtime=0 cores=0-31
/proc/31912/stat : 31912 (ppfolio) S 31910 31912 31886 0 -1 4202496 513 0 0 0 0 0 0 0 24 0 1 0 611357710 18763776 385 268435456000 4194304 4287625 140734349320608 18446744073709551615 258650975999 0 0 4096 65536 9184107106532327423 0 0 17 4 0 0 0
/proc/31912/statm: 4581 385 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100389 s]
/proc/loadavg: 27.21 31.05 31.73 2/413 31912
/proc/meminfo: memFree=256506728/264257624 swapFree=67111528/67111528
[pid=31912] ppid=31910 vsize=18324 CPUtime=0 cores=0-31
/proc/31912/stat : 31912 (ppfolio) S 31910 31912 31886 0 -1 4202496 513 0 0 0 0 0 0 0 24 0 1 0 611357710 18763776 385 268435456000 4194304 4287625 140734349320608 18446744073709551615 258650975999 0 0 4096 65536 9184107106532327423 0 0 17 4 0 0 0
/proc/31912/statm: 4581 385 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300397 s]
/proc/loadavg: 27.21 31.05 31.73 2/413 31912
/proc/meminfo: memFree=256506728/264257624 swapFree=67111528/67111528
[pid=31912] ppid=31910 vsize=18324 CPUtime=0 cores=0-31
/proc/31912/stat : 31912 (ppfolio) S 31910 31912 31886 0 -1 4202496 513 0 0 0 0 0 0 0 24 0 1 0 611357710 18763776 385 268435456000 4194304 4287625 140734349320608 18446744073709551615 258650975999 0 0 4096 65536 9184107106532327423 0 0 17 4 0 0 0
/proc/31912/statm: 4581 385 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700469 s]
/proc/loadavg: 27.21 31.05 31.73 2/413 31912
/proc/meminfo: memFree=256506728/264257624 swapFree=67111528/67111528
[pid=31912] ppid=31910 vsize=18324 CPUtime=0 cores=0-31
/proc/31912/stat : 31912 (ppfolio) S 31910 31912 31886 0 -1 4202496 513 0 0 0 0 0 0 0 24 0 1 0 611357710 18763776 385 268435456000 4194304 4287625 140734349320608 18446744073709551615 258650975999 0 0 4096 65536 9184107106532327423 0 0 17 4 0 0 0
/proc/31912/statm: 4581 385 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50138 s]
/proc/loadavg: 25.43 30.62 31.59 6/419 31923
/proc/meminfo: memFree=256334652/264257624 swapFree=67111528/67111528
[pid=31912] ppid=31910 vsize=18324 CPUtime=0 cores=0-31
/proc/31912/stat : 31912 (ppfolio) S 31910 31912 31886 0 -1 4202496 513 0 0 0 0 0 0 0 15 0 1 0 611357710 18763776 385 268435456000 4194304 4287625 140734349320608 18446744073709551615 258650975999 0 0 4096 65536 9184107106532327423 0 0 17 4 0 0 0
/proc/31912/statm: 4581 385 337 23 0 91 0
[pid=31913] ppid=31912 vsize=39408 CPUtime=1.48 cores=0
/proc/31913/stat : 31913 (cryptominisat) R 31912 31912 31886 0 -1 4202496 7792 0 0 0 146 2 0 0 20 0 1 0 611357711 40353792 5001 268435456000 4194304 4587354 140733546198256 18446744073709551615 4228273 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/31913/statm: 9852 5001 322 96 0 6414 0
[pid=31914] ppid=31912 vsize=21480 CPUtime=1.48 cores=1
/proc/31914/stat : 31914 (clasp) R 31912 31912 31886 0 -1 4202496 3108 0 0 0 147 1 0 0 20 0 1 0 611357711 21995520 2696 268435456000 4194304 4785740 140737008533584 18446744073709551615 4495618 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/31914/statm: 5370 2696 314 145 0 2415 0
[pid=31915] ppid=31912 vsize=457544 CPUtime=1.48 cores=2
/proc/31915/stat : 31915 (TNM) R 31912 31912 31886 0 -1 4202496 2012 0 0 0 147 1 0 0 20 0 1 0 611357711 468525056 1910 268435456000 134512640 134534564 4293146592 18446744073709551615 134524664 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/31915/statm: 114386 1910 91 6 0 113051 0
[pid=31916] ppid=31912 vsize=83908 CPUtime=1.48 cores=3
/proc/31916/stat : 31916 (march_hi) R 31912 31912 31886 0 -1 4202496 24062 0 0 0 142 6 0 0 20 0 1 0 611357711 85921792 16437 268435456000 4194304 4331988 140734860185456 18446744073709551615 258650616632 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/31916/statm: 20977 16437 149 34 0 19419 0
[pid=31917] ppid=31912 vsize=680208 CPUtime=5.41 cores=4-31
/proc/31917/stat : 31917 (plingeling) S 31912 31912 31886 0 -1 4202496 74329 1244 0 0 474 66 0 1 18 0 29 0 611357711 696532992 42746 268435456000 4194304 4356188 140734552527408 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 23 0 0 0
/proc/31917/statm: 170052 42746 147 40 0 168596 0
Current children cumulated CPU time (s) 11.33
Current children cumulated vsize (KiB) 1300872

[startup+3.10244 s]
/proc/loadavg: 25.43 30.62 31.59 34/447 31951
/proc/meminfo: memFree=256221220/264257624 swapFree=67111528/67111528
[pid=31912] ppid=31910 vsize=18324 CPUtime=0 cores=0-31
/proc/31912/stat : 31912 (ppfolio) S 31910 31912 31886 0 -1 4202496 513 0 0 0 0 0 0 0 15 0 1 0 611357710 18763776 385 268435456000 4194304 4287625 140734349320608 18446744073709551615 258650975999 0 0 4096 65536 9184107106532327423 0 0 17 4 0 0 0
/proc/31912/statm: 4581 385 337 23 0 91 0
[pid=31913] ppid=31912 vsize=39408 CPUtime=3.08 cores=0
/proc/31913/stat : 31913 (cryptominisat) R 31912 31912 31886 0 -1 4202496 8000 0 0 0 306 2 0 0 25 0 1 0 611357711 40353792 5206 268435456000 4194304 4587354 140733546198256 18446744073709551615 4237681 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/31913/statm: 9852 5206 325 96 0 6414 0
[pid=31914] ppid=31912 vsize=23108 CPUtime=3.08 cores=1
/proc/31914/stat : 31914 (clasp) R 31912 31912 31886 0 -1 4202496 3694 0 0 0 307 1 0 0 25 0 1 0 611357711 23662592 3111 268435456000 4194304 4785740 140737008533584 18446744073709551615 4287158 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/31914/statm: 5777 3111 338 145 0 2822 0
[pid=31915] ppid=31912 vsize=458072 CPUtime=3.08 cores=2
/proc/31915/stat : 31915 (TNM) R 31912 31912 31886 0 -1 4202496 2600 0 0 0 307 1 0 0 25 0 1 0 611357711 469065728 2498 268435456000 134512640 134534564 4293146592 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/31915/statm: 114518 2498 95 6 0 113183 0
[pid=31916] ppid=31912 vsize=192040 CPUtime=3.08 cores=3
/proc/31916/stat : 31916 (march_hi) R 31912 31912 31886 0 -1 4202496 52791 0 0 0 295 13 0 0 25 0 1 0 611357711 196648960 45166 268435456000 4194304 4331988 140734860185456 18446744073709551615 258650616632 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/31916/statm: 48010 45166 149 34 0 46452 0
[pid=31917] ppid=31912 vsize=699360 CPUtime=50.19 cores=4-31
/proc/31917/stat : 31917 (plingeling) S 31912 31912 31886 0 -1 4202496 80695 1244 0 0 4949 69 0 1 18 0 29 0 611357711 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 23 0 0 0
/proc/31917/statm: 174840 37572 159 40 0 173384 0
[pid=31917/tid=31924] ppid=31912 vsize=699360 CPUtime=1.75 cores=4-31
/proc/31917/task/31924/stat : 31924 (plingeling) R 31912 31912 31886 0 -1 4202560 1220 1244 0 0 172 2 0 1 25 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4210587 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31917/tid=31925] ppid=31912 vsize=699360 CPUtime=1.76 cores=4-31
/proc/31917/task/31925/stat : 31925 (plingeling) R 31912 31912 31886 0 -1 4202560 2068 1244 0 0 173 2 0 1 20 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31917/tid=31926] ppid=31912 vsize=699360 CPUtime=1.76 cores=4-31
/proc/31917/task/31926/stat : 31926 (plingeling) R 31912 31912 31886 0 -1 4202560 1775 1244 0 0 173 2 0 1 20 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4332020 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=31917/tid=31927] ppid=31912 vsize=699360 CPUtime=1.73 cores=4-31
/proc/31917/task/31927/stat : 31927 (plingeling) R 31912 31912 31886 0 -1 4202560 1765 1244 0 0 169 3 0 1 20 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4279765 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=31917/tid=31928] ppid=31912 vsize=699360 CPUtime=1.75 cores=4-31
/proc/31917/task/31928/stat : 31928 (plingeling) R 31912 31912 31886 0 -1 4202560 2610 1244 0 0 171 3 0 1 20 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=31917/tid=31929] ppid=31912 vsize=699360 CPUtime=1.75 cores=4-31
/proc/31917/task/31929/stat : 31929 (plingeling) R 31912 31912 31886 0 -1 4202560 2011 1244 0 0 172 2 0 1 20 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=31917/tid=31930] ppid=31912 vsize=699360 CPUtime=1.75 cores=4-31
/proc/31917/task/31930/stat : 31930 (plingeling) R 31912 31912 31886 0 -1 4202560 1336 1244 0 0 172 2 0 1 25 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331383 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=31917/tid=31931] ppid=31912 vsize=699360 CPUtime=1.75 cores=4-31
/proc/31917/task/31931/stat : 31931 (plingeling) R 31912 31912 31886 0 -1 4202560 2163 1244 0 0 172 2 0 1 20 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4234124 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=31917/tid=31932] ppid=31912 vsize=699360 CPUtime=1.76 cores=4-31
/proc/31917/task/31932/stat : 31932 (plingeling) R 31912 31912 31886 0 -1 4202560 2419 1244 0 0 173 2 0 1 20 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4244402 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=31917/tid=31933] ppid=31912 vsize=699360 CPUtime=1.76 cores=4-31
/proc/31917/task/31933/stat : 31933 (plingeling) R 31912 31912 31886 0 -1 4202560 1634 1244 0 0 173 2 0 1 20 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331383 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=31917/tid=31934] ppid=31912 vsize=699360 CPUtime=1.75 cores=4-31
/proc/31917/task/31934/stat : 31934 (plingeling) R 31912 31912 31886 0 -1 4202560 1788 1244 0 0 172 2 0 1 20 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=31917/tid=31935] ppid=31912 vsize=699360 CPUtime=1.76 cores=4-31
/proc/31917/task/31935/stat : 31935 (plingeling) R 31912 31912 31886 0 -1 4202560 1859 1244 0 0 173 2 0 1 20 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331360 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=31917/tid=31936] ppid=31912 vsize=699360 CPUtime=1.75 cores=4-31
/proc/31917/task/31936/stat : 31936 (plingeling) R 31912 31912 31886 0 -1 4202560 1472 1244 0 0 172 2 0 1 20 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31917/tid=31937] ppid=31912 vsize=699360 CPUtime=1.76 cores=4-31
/proc/31917/task/31937/stat : 31937 (plingeling) R 31912 31912 31886 0 -1 4202560 2019 1244 0 0 173 2 0 1 20 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331939 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=31917/tid=31938] ppid=31912 vsize=699360 CPUtime=1.75 cores=4-31
/proc/31917/task/31938/stat : 31938 (plingeling) R 31912 31912 31886 0 -1 4202560 1881 1244 0 0 172 2 0 1 20 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=31917/tid=31939] ppid=31912 vsize=699360 CPUtime=1.76 cores=4-31
/proc/31917/task/31939/stat : 31939 (plingeling) R 31912 31912 31886 0 -1 4202560 1459 1244 0 0 172 3 0 1 20 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4234247 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=31917/tid=31940] ppid=31912 vsize=699360 CPUtime=1.73 cores=4-31
/proc/31917/task/31940/stat : 31940 (plingeling) R 31912 31912 31886 0 -1 4202560 2002 1244 0 0 172 0 0 1 20 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4330780 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=31917/tid=31941] ppid=31912 vsize=699360 CPUtime=1.76 cores=4-31
/proc/31917/task/31941/stat : 31941 (plingeling) R 31912 31912 31886 0 -1 4202560 1786 1244 0 0 172 3 0 1 20 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=31917/tid=31942] ppid=31912 vsize=699360 CPUtime=1.72 cores=4-31
/proc/31917/task/31942/stat : 31942 (plingeling) R 31912 31912 31886 0 -1 4202560 1313 1244 0 0 171 0 0 1 25 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=31917/tid=31943] ppid=31912 vsize=699360 CPUtime=1.75 cores=4-31
/proc/31917/task/31943/stat : 31943 (plingeling) R 31912 31912 31886 0 -1 4202560 1891 1244 0 0 172 2 0 1 20 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=31917/tid=31944] ppid=31912 vsize=699360 CPUtime=1.73 cores=4-31
/proc/31917/task/31944/stat : 31944 (plingeling) R 31912 31912 31886 0 -1 4202560 1768 1244 0 0 172 0 0 1 25 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331069 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=31917/tid=31945] ppid=31912 vsize=699360 CPUtime=1.74 cores=4-31
/proc/31917/task/31945/stat : 31945 (plingeling) R 31912 31912 31886 0 -1 4202560 1154 1244 0 0 172 1 0 1 25 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331076 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=31917/tid=31946] ppid=31912 vsize=699360 CPUtime=1.74 cores=4-31
/proc/31917/task/31946/stat : 31946 (plingeling) R 31912 31912 31886 0 -1 4202560 1702 1244 0 0 172 1 0 1 25 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4330960 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=31917/tid=31947] ppid=31912 vsize=699360 CPUtime=1.74 cores=4-31
/proc/31917/task/31947/stat : 31947 (plingeling) R 31912 31912 31886 0 -1 4202560 1225 1244 0 0 172 1 0 1 25 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4330960 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=31917/tid=31948] ppid=31912 vsize=699360 CPUtime=1.66 cores=4-31
/proc/31917/task/31948/stat : 31948 (plingeling) R 31912 31912 31886 0 -1 4202560 1036 1244 0 0 165 0 0 1 23 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331047 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=31917/tid=31949] ppid=31912 vsize=699360 CPUtime=1.74 cores=4-31
/proc/31917/task/31949/stat : 31949 (plingeling) R 31912 31912 31886 0 -1 4202560 2268 1244 0 0 172 1 0 1 20 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331383 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=31917/tid=31950] ppid=31912 vsize=699360 CPUtime=1.76 cores=4-31
/proc/31917/task/31950/stat : 31950 (plingeling) R 31912 31912 31886 0 -1 4202560 2214 1244 0 0 173 2 0 1 20 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4330990 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=31917/tid=31951] ppid=31912 vsize=699360 CPUtime=1.75 cores=4-31
/proc/31917/task/31951/stat : 31951 (plingeling) R 31912 31912 31886 0 -1 4202560 2371 1244 0 0 172 2 0 1 20 0 29 0 611357844 716144640 37572 268435456000 4194304 4356188 140734552527408 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 16 0 0 0
Current children cumulated CPU time (s) 62.51
Current children cumulated vsize (KiB) 1430312

[startup+6.30156 s]
/proc/loadavg: 25.43 30.62 31.59 33/447 31951
/proc/meminfo: memFree=256064796/264257624 swapFree=67111528/67111528
[pid=31912] ppid=31910 vsize=18324 CPUtime=0 cores=0-31
/proc/31912/stat : 31912 (ppfolio) S 31910 31912 31886 0 -1 4202496 513 0 0 0 0 0 0 0 15 0 1 0 611357710 18763776 385 268435456000 4194304 4287625 140734349320608 18446744073709551615 258650975999 0 0 4096 65536 9184107106532327423 0 0 17 4 0 0 0
/proc/31912/statm: 4581 385 337 23 0 91 0
[pid=31913] ppid=31912 vsize=39408 CPUtime=6.28 cores=0
/proc/31913/stat : 31913 (cryptominisat) R 31912 31912 31886 0 -1 4202496 8674 0 0 0 626 2 0 0 25 0 1 0 611357711 40353792 5860 268435456000 4194304 4587354 140733546198256 18446744073709551615 4237665 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/31913/statm: 9852 5860 325 96 0 6414 0
[pid=31914] ppid=31912 vsize=29164 CPUtime=6.28 cores=1
/proc/31914/stat : 31914 (clasp) R 31912 31912 31886 0 -1 4202496 5148 0 0 0 627 1 0 0 25 0 1 0 611357711 29863936 4565 268435456000 4194304 4785740 140737008533584 18446744073709551615 4287207 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/31914/statm: 7291 4565 338 145 0 4336 0
[pid=31915] ppid=31912 vsize=458072 CPUtime=6.28 cores=2
/proc/31915/stat : 31915 (TNM) R 31912 31912 31886 0 -1 4202496 2600 0 0 0 627 1 0 0 25 0 1 0 611357711 469065728 2498 268435456000 134512640 134534564 4293146592 18446744073709551615 134529640 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/31915/statm: 114518 2498 95 6 0 113183 0
[pid=31916] ppid=31912 vsize=164120 CPUtime=6.28 cores=3
/proc/31916/stat : 31916 (march_hi) R 31912 31912 31886 0 -1 4202496 88253 0 0 0 606 22 0 0 25 0 1 0 611357711 168058880 34615 268435456000 4194304 4331988 140734860185456 18446744073709551615 258650604641 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/31916/statm: 41030 34615 149 34 0 39472 0
[pid=31917] ppid=31912 vsize=741884 CPUtime=139.64 cores=4-31
/proc/31917/stat : 31917 (plingeling) S 31912 31912 31886 0 -1 4202496 101080 1244 0 0 13890 73 0 1 18 0 29 0 611357711 759689216 49879 268435456000 4194304 4356188 140734552527408 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 23 0 0 0
/proc/31917/statm: 185471 49879 159 40 0 184015 0
[pid=31917/tid=31924] ppid=31912 vsize=741884 CPUtime=4.95 cores=4-31
/proc/31917/task/31924/stat : 31924 (plingeling) R 31912 31912 31886 0 -1 4202560 1412 1244 0 0 492 2 0 1 25 0 29 0 611357844 759689216 49879 268435456000 4194304 4356188 140734552527408 18446744073709551615 4279602 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31917/tid=31925] ppid=31912 vsize=741884 CPUtime=4.95 cores=4-31
/proc/31917/task/31925/stat : 31925 (plingeling) R 31912 31912 31886 0 -1 4202560 3033 1244 0 0 492 2 0 1 25 0 29 0 611357844 759689216 49879 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331055 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31917/tid=31926] ppid=31912 vsize=741884 CPUtime=4.96 cores=4-31
/proc/31917/task/31926/stat : 31926 (plingeling) R 31912 31912 31886 0 -1 4202560 1903 1244 0 0 493 2 0 1 25 0 29 0 611357844 759689216 49879 268435456000 4194304 4356188 140734552527408 18446744073709551615 4330862 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=31917/tid=31927] ppid=31912 vsize=741884 CPUtime=4.83 cores=4-31
/proc/31917/task/31927/stat : 31927 (plingeling) R 31912 31912 31886 0 -1 4202560 3318 1244 0 0 479 3 0 1 25 0 29 0 611357844 759689216 49879 268435456000 4194304 4356188 140734552527408 18446744073709551615 4281515 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=31917/tid=31928] ppid=31912 vsize=741884 CPUtime=4.95 cores=4-31
/proc/31917/task/31928/stat : 31928 (plingeling) R 31912 31912 31886 0 -1 4202560 2642 1244 0 0 491 3 0 1 25 0 29 0 611357844 759689216 49879 268435456000 4194304 4356188 140734552527408 18446744073709551615 4330960 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=31917/tid=31929] ppid=31912 vsize=741884 CPUtime=4.95 cores=4-31

################
# More data... #
################

[pid=31915] ppid=31912 vsize=458072 CPUtime=4961.34 cores=2
/proc/31915/stat : 31915 (TNM) R 31912 31912 31886 0 -1 4202496 2600 0 0 0 496133 1 0 0 25 0 1 0 611357711 469065728 2498 268435456000 134512640 134534564 4293146592 18446744073709551615 134530320 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/31915/statm: 114518 2498 95 6 0 113183 0
[pid=31916] ppid=31912 vsize=261796 CPUtime=4961.35 cores=3
/proc/31916/stat : 31916 (march_hi) R 31912 31912 31886 0 -1 4202496 114972 0 0 0 496088 47 0 0 25 0 1 0 611357711 268079104 61334 268435456000 4194304 4331988 140734860185456 18446744073709551615 4299264 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/31916/statm: 65449 61334 174 34 0 63891 0
[pid=31917] ppid=31912 vsize=2714276 CPUtime=138734 cores=4-31
/proc/31917/stat : 31917 (plingeling) S 31912 31912 31886 0 -1 4202496 5334522 1244 0 0 13869695 3741 0 1 18 0 29 0 611357711 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 23 0 0 0
/proc/31917/statm: 678569 555319 159 40 0 677113 0
[pid=31917/tid=31924] ppid=31912 vsize=2714276 CPUtime=4959.94 cores=4-31
/proc/31917/task/31924/stat : 31924 (plingeling) R 31912 31912 31886 0 -1 4202560 151683 1244 0 0 495892 101 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31917/tid=31925] ppid=31912 vsize=2714276 CPUtime=4960.03 cores=4-31
/proc/31917/task/31925/stat : 31925 (plingeling) R 31912 31912 31886 0 -1 4202560 126990 1244 0 0 495906 96 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31917/tid=31926] ppid=31912 vsize=2714276 CPUtime=4959.89 cores=4-31
/proc/31917/task/31926/stat : 31926 (plingeling) R 31912 31912 31886 0 -1 4202560 171257 1244 0 0 495875 113 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=31917/tid=31927] ppid=31912 vsize=2714276 CPUtime=4944.74 cores=4-31
/proc/31917/task/31927/stat : 31927 (plingeling) R 31912 31912 31886 0 -1 4202560 197585 1244 0 0 494343 130 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331045 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=31917/tid=31928] ppid=31912 vsize=2714276 CPUtime=4925.84 cores=4-31
/proc/31917/task/31928/stat : 31928 (plingeling) R 31912 31912 31886 0 -1 4202560 130920 1244 0 0 492516 67 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=31917/tid=31929] ppid=31912 vsize=2714276 CPUtime=4959.8 cores=4-31
/proc/31917/task/31929/stat : 31929 (plingeling) R 31912 31912 31886 0 -1 4202560 187314 1244 0 0 495848 131 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=31917/tid=31930] ppid=31912 vsize=2714276 CPUtime=4957.58 cores=4-31
/proc/31917/task/31930/stat : 31930 (plingeling) R 31912 31912 31886 0 -1 4202560 139433 1244 0 0 495607 150 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=31917/tid=31931] ppid=31912 vsize=2714276 CPUtime=4960.01 cores=4-31
/proc/31917/task/31931/stat : 31931 (plingeling) R 31912 31912 31886 0 -1 4202560 158037 1244 0 0 495807 193 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=31917/tid=31932] ppid=31912 vsize=2714276 CPUtime=4959.96 cores=4-31
/proc/31917/task/31932/stat : 31932 (plingeling) R 31912 31912 31886 0 -1 4202560 392051 1244 0 0 495773 222 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4211007 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=31917/tid=31933] ppid=31912 vsize=2714276 CPUtime=4960.03 cores=4-31
/proc/31917/task/31933/stat : 31933 (plingeling) R 31912 31912 31886 0 -1 4202560 187121 1244 0 0 495875 127 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=31917/tid=31934] ppid=31912 vsize=2714276 CPUtime=4959.81 cores=4-31
/proc/31917/task/31934/stat : 31934 (plingeling) R 31912 31912 31886 0 -1 4202560 115241 1244 0 0 495905 75 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4279478 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=31917/tid=31935] ppid=31912 vsize=2714276 CPUtime=4959.9 cores=4-31
/proc/31917/task/31935/stat : 31935 (plingeling) R 31912 31912 31886 0 -1 4202560 231669 1244 0 0 495846 143 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=31917/tid=31936] ppid=31912 vsize=2714276 CPUtime=4956.06 cores=4-31
/proc/31917/task/31936/stat : 31936 (plingeling) R 31912 31912 31886 0 -1 4202560 200273 1244 0 0 495517 88 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331296 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=31917/tid=31937] ppid=31912 vsize=2714276 CPUtime=4959.97 cores=4-31
/proc/31917/task/31937/stat : 31937 (plingeling) R 31912 31912 31886 0 -1 4202560 118100 1244 0 0 495926 70 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331002 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=31917/tid=31938] ppid=31912 vsize=2714276 CPUtime=4959.92 cores=4-31
/proc/31917/task/31938/stat : 31938 (plingeling) R 31912 31912 31886 0 -1 4202560 385918 1244 0 0 495638 353 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331069 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=31917/tid=31939] ppid=31912 vsize=2714276 CPUtime=4953.95 cores=4-31
/proc/31917/task/31939/stat : 31939 (plingeling) R 31912 31912 31886 0 -1 4202560 330228 1244 0 0 495083 311 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=31917/tid=31940] ppid=31912 vsize=2714276 CPUtime=4959.94 cores=4-31
/proc/31917/task/31940/stat : 31940 (plingeling) R 31912 31912 31886 0 -1 4202560 123651 1244 0 0 495884 109 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331296 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=31917/tid=31941] ppid=31912 vsize=2714276 CPUtime=4959.19 cores=4-31
/proc/31917/task/31941/stat : 31941 (plingeling) R 31912 31912 31886 0 -1 4202560 154795 1244 0 0 495808 110 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331340 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31917/tid=31942] ppid=31912 vsize=2714276 CPUtime=4954.59 cores=4-31
/proc/31917/task/31942/stat : 31942 (plingeling) R 31912 31912 31886 0 -1 4202560 191317 1244 0 0 495332 126 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331055 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=31917/tid=31943] ppid=31912 vsize=2714276 CPUtime=4922.92 cores=4-31
/proc/31917/task/31943/stat : 31943 (plingeling) R 31912 31912 31886 0 -1 4202560 179556 1244 0 0 492170 121 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331015 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=31917/tid=31944] ppid=31912 vsize=2714276 CPUtime=4959.84 cores=4-31
/proc/31917/task/31944/stat : 31944 (plingeling) R 31912 31912 31886 0 -1 4202560 197674 1244 0 0 495860 123 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331371 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=31917/tid=31945] ppid=31912 vsize=2714276 CPUtime=4959.9 cores=4-31
/proc/31917/task/31945/stat : 31945 (plingeling) R 31912 31912 31886 0 -1 4202560 167352 1244 0 0 495887 102 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=31917/tid=31946] ppid=31912 vsize=2714276 CPUtime=4959.99 cores=4-31
/proc/31917/task/31946/stat : 31946 (plingeling) R 31912 31912 31886 0 -1 4202560 95164 1244 0 0 495965 33 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4330976 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=31917/tid=31947] ppid=31912 vsize=2714276 CPUtime=4932.5 cores=4-31
/proc/31917/task/31947/stat : 31947 (plingeling) R 31912 31912 31886 0 -1 4202560 370309 1244 0 0 492928 321 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=31917/tid=31948] ppid=31912 vsize=2714276 CPUtime=4959.82 cores=4-31
/proc/31917/task/31948/stat : 31948 (plingeling) R 31912 31912 31886 0 -1 4202560 136178 1244 0 0 495883 98 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331193 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=31917/tid=31949] ppid=31912 vsize=2714276 CPUtime=4959.83 cores=4-31
/proc/31917/task/31949/stat : 31949 (plingeling) R 31912 31912 31886 0 -1 4202560 125731 1244 0 0 495927 55 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=31917/tid=31950] ppid=31912 vsize=2714276 CPUtime=4948.74 cores=4-31
/proc/31917/task/31950/stat : 31950 (plingeling) R 31912 31912 31886 0 -1 4202560 195375 1244 0 0 494764 109 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4332133 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=31917/tid=31951] ppid=31912 vsize=2714276 CPUtime=4958.4 cores=4-31
/proc/31917/task/31951/stat : 31951 (plingeling) R 31912 31912 31886 0 -1 4202560 143114 1244 0 0 495795 44 0 1 25 0 29 0 611357844 2779418624 555319 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331074 0 0 4096 17442 0 0 0 -1 16 0 0 0
Current children cumulated CPU time (s) 158580
Current children cumulated vsize (KiB) 5406732



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5006.8 s]
/proc/loadavg: 32.07 32.05 32.07 33/447 32106
/proc/meminfo: memFree=253742920/264257624 swapFree=67111528/67111528
[pid=31912] ppid=31910 vsize=18324 CPUtime=0 cores=0-31
/proc/31912/stat : 31912 (ppfolio) S 31910 31912 31886 0 -1 4202496 513 0 0 0 0 0 0 0 15 0 1 0 611357710 18763776 385 268435456000 4194304 4287625 140734349320608 18446744073709551615 258650975999 0 0 4096 65536 9184107106532327423 0 0 17 4 0 0 0
/proc/31912/statm: 4581 385 337 23 0 91 0
[pid=31913] ppid=31912 vsize=1767776 CPUtime=5005.86 cores=0
/proc/31913/stat : 31913 (cryptominisat) R 31912 31912 31886 0 -1 4202496 650565 0 0 0 500261 325 0 0 25 0 1 0 611357711 1810202624 25960 268435456000 4194304 4587354 140733546198256 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/31913/statm: 441944 25960 335 96 0 438506 0
[pid=31914] ppid=31912 vsize=186488 CPUtime=5005.8 cores=1
/proc/31914/stat : 31914 (clasp) R 31912 31912 31886 0 -1 4202496 41401 0 0 0 500563 17 0 0 25 0 1 0 611357711 190963712 40271 268435456000 4194304 4785740 140737008533584 18446744073709551615 4287204 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/31914/statm: 46622 40271 339 145 0 43667 0
[pid=31915] ppid=31912 vsize=458072 CPUtime=5005.86 cores=2
/proc/31915/stat : 31915 (TNM) R 31912 31912 31886 0 -1 4202496 2600 0 0 0 500585 1 0 0 25 0 1 0 611357711 469065728 2498 268435456000 134512640 134534564 4293146592 18446744073709551615 134530379 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/31915/statm: 114518 2498 95 6 0 113183 0
[pid=31916] ppid=31912 vsize=261796 CPUtime=5005.86 cores=3
/proc/31916/stat : 31916 (march_hi) R 31912 31912 31886 0 -1 4202496 115101 0 0 0 500539 47 0 0 25 0 1 0 611357711 268079104 61463 268435456000 4194304 4331988 140734860185456 18446744073709551615 4226242 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/31916/statm: 65449 61463 174 34 0 63891 0
[pid=31917] ppid=31912 vsize=2734764 CPUtime=139979 cores=4-31
/proc/31917/stat : 31917 (plingeling) S 31912 31912 31886 0 -1 4202496 5355900 1244 0 0 13994182 3758 0 1 18 0 29 0 611357711 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 23 0 0 0
/proc/31917/statm: 683691 560441 159 40 0 682235 0
[pid=31917/tid=31924] ppid=31912 vsize=2734764 CPUtime=5004.45 cores=4-31
/proc/31917/task/31924/stat : 31924 (plingeling) R 31912 31912 31886 0 -1 4202560 151683 1244 0 0 500343 101 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31917/tid=31925] ppid=31912 vsize=2734764 CPUtime=5004.54 cores=4-31
/proc/31917/task/31925/stat : 31925 (plingeling) R 31912 31912 31886 0 -1 4202560 126990 1244 0 0 500357 96 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31917/tid=31926] ppid=31912 vsize=2734764 CPUtime=5004.4 cores=4-31
/proc/31917/task/31926/stat : 31926 (plingeling) R 31912 31912 31886 0 -1 4202560 171513 1244 0 0 500326 113 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331002 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=31917/tid=31927] ppid=31912 vsize=2734764 CPUtime=4987.99 cores=4-31
/proc/31917/task/31927/stat : 31927 (plingeling) R 31912 31912 31886 0 -1 4202560 199634 1244 0 0 498666 132 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=31917/tid=31928] ppid=31912 vsize=2734764 CPUtime=4970.35 cores=4-31
/proc/31917/task/31928/stat : 31928 (plingeling) R 31912 31912 31886 0 -1 4202560 132968 1244 0 0 496967 67 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331151 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=31917/tid=31929] ppid=31912 vsize=2734764 CPUtime=5004.31 cores=4-31
/proc/31917/task/31929/stat : 31929 (plingeling) R 31912 31912 31886 0 -1 4202560 187314 1244 0 0 500299 131 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=31917/tid=31930] ppid=31912 vsize=2734764 CPUtime=5002.07 cores=4-31
/proc/31917/task/31930/stat : 31930 (plingeling) R 31912 31912 31886 0 -1 4202560 139433 1244 0 0 500055 151 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331212 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=31917/tid=31931] ppid=31912 vsize=2734764 CPUtime=5004.52 cores=4-31
/proc/31917/task/31931/stat : 31931 (plingeling) R 31912 31912 31886 0 -1 4202560 158037 1244 0 0 500258 193 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=31917/tid=31932] ppid=31912 vsize=2734764 CPUtime=5004.47 cores=4-31
/proc/31917/task/31932/stat : 31932 (plingeling) R 31912 31912 31886 0 -1 4202560 392819 1244 0 0 500224 222 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331055 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=31917/tid=31933] ppid=31912 vsize=2734764 CPUtime=5004.54 cores=4-31
/proc/31917/task/31933/stat : 31933 (plingeling) R 31912 31912 31886 0 -1 4202560 187121 1244 0 0 500326 127 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331345 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=31917/tid=31934] ppid=31912 vsize=2734764 CPUtime=5004.33 cores=4-31
/proc/31917/task/31934/stat : 31934 (plingeling) R 31912 31912 31886 0 -1 4202560 115241 1244 0 0 500356 76 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4332007 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=31917/tid=31935] ppid=31912 vsize=2734764 CPUtime=5004.41 cores=4-31
/proc/31917/task/31935/stat : 31935 (plingeling) R 31912 31912 31886 0 -1 4202560 231669 1244 0 0 500297 143 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4330992 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=31917/tid=31936] ppid=31912 vsize=2734764 CPUtime=5000.58 cores=4-31
/proc/31917/task/31936/stat : 31936 (plingeling) R 31912 31912 31886 0 -1 4202560 200273 1244 0 0 499969 88 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331028 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=31917/tid=31937] ppid=31912 vsize=2734764 CPUtime=5004.48 cores=4-31
/proc/31917/task/31937/stat : 31937 (plingeling) R 31912 31912 31886 0 -1 4202560 118100 1244 0 0 500377 70 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=31917/tid=31938] ppid=31912 vsize=2734764 CPUtime=5004.43 cores=4-31
/proc/31917/task/31938/stat : 31938 (plingeling) R 31912 31912 31886 0 -1 4202560 387454 1244 0 0 500089 353 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4330999 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=31917/tid=31939] ppid=31912 vsize=2734764 CPUtime=4998.45 cores=4-31
/proc/31917/task/31939/stat : 31939 (plingeling) R 31912 31912 31886 0 -1 4202560 332277 1244 0 0 499533 311 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=31917/tid=31940] ppid=31912 vsize=2734764 CPUtime=5004.45 cores=4-31
/proc/31917/task/31940/stat : 31940 (plingeling) R 31912 31912 31886 0 -1 4202560 124035 1244 0 0 500335 109 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331151 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=31917/tid=31941] ppid=31912 vsize=2734764 CPUtime=5003.7 cores=4-31
/proc/31917/task/31941/stat : 31941 (plingeling) R 31912 31912 31886 0 -1 4202560 154795 1244 0 0 500259 110 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331069 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31917/tid=31942] ppid=31912 vsize=2734764 CPUtime=4999.1 cores=4-31
/proc/31917/task/31942/stat : 31942 (plingeling) R 31912 31912 31886 0 -1 4202560 191317 1244 0 0 499783 126 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331045 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=31917/tid=31943] ppid=31912 vsize=2734764 CPUtime=4967.42 cores=4-31
/proc/31917/task/31943/stat : 31943 (plingeling) R 31912 31912 31886 0 -1 4202560 181604 1244 0 0 496620 121 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=31917/tid=31944] ppid=31912 vsize=2734764 CPUtime=5004.35 cores=4-31
/proc/31917/task/31944/stat : 31944 (plingeling) R 31912 31912 31886 0 -1 4202560 197674 1244 0 0 500311 123 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=31917/tid=31945] ppid=31912 vsize=2734764 CPUtime=5004.42 cores=4-31
/proc/31917/task/31945/stat : 31945 (plingeling) R 31912 31912 31886 0 -1 4202560 169400 1244 0 0 500334 107 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=31917/tid=31946] ppid=31912 vsize=2734764 CPUtime=5004.51 cores=4-31
/proc/31917/task/31946/stat : 31946 (plingeling) R 31912 31912 31886 0 -1 4202560 99260 1244 0 0 500416 34 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331578 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=31917/tid=31947] ppid=31912 vsize=2734764 CPUtime=4977.01 cores=4-31
/proc/31917/task/31947/stat : 31947 (plingeling) R 31912 31912 31886 0 -1 4202560 371333 1244 0 0 497379 321 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=31917/tid=31948] ppid=31912 vsize=2734764 CPUtime=5004.33 cores=4-31
/proc/31917/task/31948/stat : 31948 (plingeling) R 31912 31912 31886 0 -1 4202560 136178 1244 0 0 500334 98 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=31917/tid=31949] ppid=31912 vsize=2734764 CPUtime=5004.34 cores=4-31
/proc/31917/task/31949/stat : 31949 (plingeling) R 31912 31912 31886 0 -1 4202560 125731 1244 0 0 500378 55 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331984 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=31917/tid=31950] ppid=31912 vsize=2734764 CPUtime=4993.25 cores=4-31
/proc/31917/task/31950/stat : 31950 (plingeling) R 31912 31912 31886 0 -1 4202560 197679 1244 0 0 499210 114 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4332003 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=31917/tid=31951] ppid=31912 vsize=2734764 CPUtime=5002.9 cores=4-31
/proc/31917/task/31951/stat : 31951 (plingeling) R 31912 31912 31886 0 -1 4202560 143882 1244 0 0 500245 44 0 1 25 0 29 0 611357844 2800398336 560441 268435456000 4194304 4356188 140734552527408 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 16 0 0 0
Current children cumulated CPU time (s) 160003
Current children cumulated vsize (KiB) 5427220

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 31912 and gives
#  childrusage.ru_utime.tv_sec=10008
#  childrusage.ru_utime.tv_usec=478480
#  childrusage.ru_stime.tv_sec=3
#  childrusage.ru_stime.tv_usec=289499
# CPU time returned by wait4() is 10011.8
# while last known CPU time is 160003
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5006.84
CPU time (s): 160003
CPU user time (s): 159961
CPU system time (s): 41.49
CPU usage (%): 3195.69
Max. virtual memory (cumulated for all children) (KiB): 7045172

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 10008.5
system time used= 3.2895
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 653679
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 660
involuntary context switches= 16129

runsolver used 28.6256 second user time and 112.742 second system time

The end

Launcher Data 

Begin job on node200 at 2011-05-10 05:38:33
IDJOB=3347309
IDBENCH=71089
IDSOLVER=1808
FILE ID=node200/3347309-1304998713
RUNJOBID= node200-1304998713-31900
PBS_JOBID= 13282031
Free space on /tmp= 62048 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT09/APPLICATIONS/satComp09_BioInstances/rpoc_xits_09_UNSAT.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3347309-1304998713/watcher-3347309-1304998713 -o /tmp/evaluation-result-3347309-1304998713/solver-3347309-1304998713 -C 160000 -W 5100 -M 266338304  ppfolio HOME/instance-3347309-1304998713.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 266338304 MiB
NBCORE= 32

MD5SUM BENCH= feb620c17f2a4d7899a2e7883baa7391
RANDOM SEED=1796685082

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.98
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.23
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      256507132 kB
Buffers:        373080 kB
Cached:        7036840 kB
SwapCached:          0 kB
Active:        6553920 kB
Inactive:       924528 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      256507132 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           11708 kB
Writeback:           0 kB
AnonPages:       68748 kB
Mapped:          14420 kB
Slab:           193824 kB
PageTables:       4544 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   185272 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 62040 MiB
End job on node200 at 2011-05-10 07:02:02