Trace number 3347363

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 160001 5012.24

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
openstacks-sequencedstrips-nonadl-nonnegated-os-sequencedstrips-p30_3.035-NOTKNOWN.cnf
MD5SUMc4319228342e10028de028a9b67c3404
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables133566
Number of clauses676901
Sum of the clauses size1609791
Maximum clause length63
Minimum clause length1
Number of clauses of size 1211
Number of clauses of size 2668220
Number of clauses of size 32100
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 56370

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 32 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3347363-1305059596/watcher-3347363-1305059596 -o /tmp/evaluation-result-3347363-1305059596/solver-3347363-1305059596 -C 160000 -W 5100 -M 266338304 ppfolio HOME/instance-3347363-1305059596.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 272730423296 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 272730474496 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 25.02 30.50 31.50 1/413 4977
/proc/meminfo: memFree=256237276/264257624 swapFree=67111528/67111528
[pid=4977] ppid=4975 vsize=18324 CPUtime=0 cores=0-31
/proc/4977/stat : 4977 (ppfolio) S 4975 4977 4951 0 -1 4202496 515 0 0 0 0 0 0 0 23 0 1 0 617446088 18763776 386 268435456000 4194304 4287625 140736406997040 18446744073709551615 258650975999 0 0 4096 65536 9197450367330156543 0 0 17 12 0 0 0
/proc/4977/statm: 4581 386 337 23 0 91 0

[startup+0.129866 s]
/proc/loadavg: 25.02 30.50 31.50 1/413 4977
/proc/meminfo: memFree=256237276/264257624 swapFree=67111528/67111528
[pid=4977] ppid=4975 vsize=18324 CPUtime=0 cores=0-31
/proc/4977/stat : 4977 (ppfolio) S 4975 4977 4951 0 -1 4202496 515 0 0 0 0 0 0 0 23 0 1 0 617446088 18763776 386 268435456000 4194304 4287625 140736406997040 18446744073709551615 258650975999 0 0 4096 65536 9197450367330156543 0 0 17 12 0 0 0
/proc/4977/statm: 4581 386 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.200871 s]
/proc/loadavg: 25.02 30.50 31.50 1/413 4977
/proc/meminfo: memFree=256237276/264257624 swapFree=67111528/67111528
[pid=4977] ppid=4975 vsize=18324 CPUtime=0 cores=0-31
/proc/4977/stat : 4977 (ppfolio) S 4975 4977 4951 0 -1 4202496 515 0 0 0 0 0 0 0 23 0 1 0 617446088 18763776 386 268435456000 4194304 4287625 140736406997040 18446744073709551615 258650975999 0 0 4096 65536 9197450367330156543 0 0 17 12 0 0 0
/proc/4977/statm: 4581 386 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.30087 s]
/proc/loadavg: 25.02 30.50 31.50 1/413 4977
/proc/meminfo: memFree=256237276/264257624 swapFree=67111528/67111528
[pid=4977] ppid=4975 vsize=18324 CPUtime=0 cores=0-31
/proc/4977/stat : 4977 (ppfolio) S 4975 4977 4951 0 -1 4202496 515 0 0 0 0 0 0 0 23 0 1 0 617446088 18763776 386 268435456000 4194304 4287625 140736406997040 18446744073709551615 258650975999 0 0 4096 65536 9197450367330156543 0 0 17 12 0 0 0
/proc/4977/statm: 4581 386 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700903 s]
/proc/loadavg: 25.02 30.50 31.50 1/413 4977
/proc/meminfo: memFree=256237276/264257624 swapFree=67111528/67111528
[pid=4977] ppid=4975 vsize=18324 CPUtime=0 cores=0-31
/proc/4977/stat : 4977 (ppfolio) S 4975 4977 4951 0 -1 4202496 515 0 0 0 0 0 0 0 20 0 1 0 617446088 18763776 386 268435456000 4194304 4287625 140736406997040 18446744073709551615 258650975999 0 0 4096 65536 9197450367330156543 0 0 17 12 0 0 0
/proc/4977/statm: 4581 386 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50091 s]
/proc/loadavg: 25.02 30.50 31.50 6/419 4988
/proc/meminfo: memFree=255945332/264257624 swapFree=67111528/67111528
[pid=4977] ppid=4975 vsize=18324 CPUtime=0 cores=0-31
/proc/4977/stat : 4977 (ppfolio) S 4975 4977 4951 0 -1 4202496 515 0 0 0 0 0 0 0 15 0 1 0 617446088 18763776 386 268435456000 4194304 4287625 140736406997040 18446744073709551615 258650975999 0 0 4096 65536 9197450367330156543 0 0 17 12 0 0 0
/proc/4977/statm: 4581 386 337 23 0 91 0
[pid=4978] ppid=4977 vsize=93260 CPUtime=1.48 cores=0
/proc/4978/stat : 4978 (cryptominisat) R 4977 4977 4951 0 -1 4202496 17711 0 0 0 144 4 0 0 25 0 1 0 617446088 95498240 17364 268435456000 4194304 4587354 140735275698096 18446744073709551615 4248831 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/4978/statm: 23315 17364 310 96 0 19877 0
[pid=4979] ppid=4977 vsize=61932 CPUtime=1.48 cores=1
/proc/4979/stat : 4979 (clasp) R 4977 4977 4951 0 -1 4202496 18186 0 0 0 142 6 0 0 20 0 1 0 617446088 63418368 12536 268435456000 4194304 4785740 140736924365072 18446744073709551615 4319139 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4979/statm: 15483 12536 337 145 0 12528 0
[pid=4980] ppid=4977 vsize=489752 CPUtime=1.49 cores=2
/proc/4980/stat : 4980 (TNM) R 4977 4977 4951 0 -1 4202496 13009 0 0 0 145 4 0 0 20 0 1 0 617446089 501506048 12907 268435456000 134512640 134534564 4290360448 18446744073709551615 134524568 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/4980/statm: 122438 12907 91 6 0 121103 0
[pid=4981] ppid=4977 vsize=91156 CPUtime=1.48 cores=3
/proc/4981/stat : 4981 (march_hi) R 4977 4977 4951 0 -1 4194304 27940 0 0 0 140 8 0 0 20 0 1 0 617446089 93343744 14315 268435456000 4194304 4331988 140736480980336 18446744073709551615 258650613936 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/4981/statm: 22789 14315 127 34 0 21231 0
[pid=4982] ppid=4977 vsize=150580 CPUtime=1.48 cores=4-31
/proc/4982/stat : 4982 (plingeling) R 4977 4977 4951 0 -1 4202496 36403 1239 0 0 140 8 0 0 21 0 1 0 617446089 154193920 36288 268435456000 4194304 4356188 140734849797200 18446744073709551615 4213270 0 0 4096 17442 0 0 0 17 16 0 0 0
/proc/4982/statm: 37645 36288 129 40 0 36189 0
Current children cumulated CPU time (s) 7.41
Current children cumulated vsize (KiB) 905004

[startup+3.10096 s]
/proc/loadavg: 25.02 30.50 31.50 6/419 4988
/proc/meminfo: memFree=255787772/264257624 swapFree=67111528/67111528
[pid=4977] ppid=4975 vsize=18324 CPUtime=0 cores=0-31
/proc/4977/stat : 4977 (ppfolio) S 4975 4977 4951 0 -1 4202496 515 0 0 0 0 0 0 0 15 0 1 0 617446088 18763776 386 268435456000 4194304 4287625 140736406997040 18446744073709551615 258650975999 0 0 4096 65536 9197450367330156543 0 0 17 12 0 0 0
/proc/4977/statm: 4581 386 337 23 0 91 0
[pid=4978] ppid=4977 vsize=93260 CPUtime=3.08 cores=0
/proc/4978/stat : 4978 (cryptominisat) R 4977 4977 4951 0 -1 4202496 17715 0 0 0 304 4 0 0 25 0 1 0 617446088 95498240 17368 268435456000 4194304 4587354 140735275698096 18446744073709551615 4421069 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/4978/statm: 23315 17368 310 96 0 19877 0
[pid=4979] ppid=4977 vsize=67872 CPUtime=3.08 cores=1
/proc/4979/stat : 4979 (clasp) R 4977 4977 4951 0 -1 4202496 19709 0 0 0 302 6 0 0 25 0 1 0 617446088 69500928 14059 268435456000 4194304 4785740 140736924365072 18446744073709551615 258650616632 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4979/statm: 16968 14059 338 145 0 14013 0
[pid=4980] ppid=4977 vsize=509684 CPUtime=3.08 cores=2
/proc/4980/stat : 4980 (TNM) R 4977 4977 4951 0 -1 4202496 23080 0 0 0 303 5 0 0 25 0 1 0 617446089 521916416 22978 268435456000 134512640 134534564 4290360448 18446744073709551615 134529957 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/4980/statm: 127421 22978 95 6 0 126086 0
[pid=4981] ppid=4977 vsize=93864 CPUtime=3.09 cores=3
/proc/4981/stat : 4981 (march_hi) R 4977 4977 4951 0 -1 4194304 56155 0 0 0 294 15 0 0 25 0 1 0 617446089 96116736 15419 268435456000 4194304 4331988 140736480980336 18446744073709551615 4261831 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/4981/statm: 23466 15419 127 34 0 21908 0
[pid=4982] ppid=4977 vsize=297156 CPUtime=3.07 cores=4-31
/proc/4982/stat : 4982 (plingeling) R 4977 4977 4951 0 -1 4202496 73111 1239 0 0 293 14 0 0 25 0 1 0 617446089 304287744 72948 268435456000 4194304 4356188 140734849797200 18446744073709551615 4210557 0 0 4096 17442 0 0 0 17 16 0 0 0
/proc/4982/statm: 74289 72948 129 40 0 72833 0
Current children cumulated CPU time (s) 15.4
Current children cumulated vsize (KiB) 1080160

[startup+6.30111 s]
/proc/loadavg: 23.42 30.08 31.36 6/419 4988
/proc/meminfo: memFree=255548372/264257624 swapFree=67111528/67111528
[pid=4977] ppid=4975 vsize=18324 CPUtime=0 cores=0-31
/proc/4977/stat : 4977 (ppfolio) S 4975 4977 4951 0 -1 4202496 515 0 0 0 0 0 0 0 15 0 1 0 617446088 18763776 386 268435456000 4194304 4287625 140736406997040 18446744073709551615 258650975999 0 0 4096 65536 9197450367330156543 0 0 17 12 0 0 0
/proc/4977/statm: 4581 386 337 23 0 91 0
[pid=4978] ppid=4977 vsize=120924 CPUtime=6.29 cores=0
/proc/4978/stat : 4978 (cryptominisat) R 4977 4977 4951 0 -1 4202496 32561 0 0 0 621 8 0 0 25 0 1 0 617446088 123826176 24168 268435456000 4194304 4587354 140735275698096 18446744073709551615 4236856 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/4978/statm: 30231 24168 328 96 0 26793 0
[pid=4979] ppid=4977 vsize=108704 CPUtime=6.28 cores=1
/proc/4979/stat : 4979 (clasp) R 4977 4977 4951 0 -1 4202496 29906 0 0 0 620 8 0 0 25 0 1 0 617446088 111312896 24256 268435456000 4194304 4785740 140736924365072 18446744073709551615 4278600 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4979/statm: 27176 24256 338 145 0 24221 0
[pid=4980] ppid=4977 vsize=509684 CPUtime=6.28 cores=2
/proc/4980/stat : 4980 (TNM) R 4977 4977 4951 0 -1 4202496 23080 0 0 0 623 5 0 0 25 0 1 0 617446089 521916416 22978 268435456000 134512640 134534564 4290360448 18446744073709551615 134528710 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/4980/statm: 127421 22978 95 6 0 126086 0
[pid=4981] ppid=4977 vsize=93428 CPUtime=6.29 cores=3
/proc/4981/stat : 4981 (march_hi) R 4977 4977 4951 0 -1 4194304 102322 0 0 0 604 25 0 0 25 0 1 0 617446089 95670272 15311 268435456000 4194304 4331988 140736480980336 18446744073709551615 4261654 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/4981/statm: 23357 15311 127 34 0 21799 0
[pid=4982] ppid=4977 vsize=490960 CPUtime=6.27 cores=4-31
/proc/4982/stat : 4982 (plingeling) R 4977 4977 4951 0 -1 4202496 121690 1239 0 0 603 24 0 0 25 0 1 0 617446089 502743040 121399 268435456000 4194304 4356188 140734849797200 18446744073709551615 4213242 0 0 4096 17442 0 0 0 17 16 0 0 0
/proc/4982/statm: 122740 121399 129 40 0 121284 0
Current children cumulated CPU time (s) 31.41
Current children cumulated vsize (KiB) 1342024

[startup+12.7022 s]
/proc/loadavg: 24.18 30.13 31.37 33/447 5016
/proc/meminfo: memFree=255212492/264257624 swapFree=67111528/67111528
[pid=4977] ppid=4975 vsize=18324 CPUtime=0 cores=0-31
/proc/4977/stat : 4977 (ppfolio) S 4975 4977 4951 0 -1 4202496 515 0 0 0 0 0 0 0 15 0 1 0 617446088 18763776 386 268435456000 4194304 4287625 140736406997040 18446744073709551615 258650975999 0 0 4096 65536 9197450367330156543 0 0 17 12 0 0 0
/proc/4977/statm: 4581 386 337 23 0 91 0
[pid=4978] ppid=4977 vsize=120924 CPUtime=12.69 cores=0
/proc/4978/stat : 4978 (cryptominisat) R 4977 4977 4951 0 -1 4202496 34245 0 0 0 1261 8 0 0 25 0 1 0 617446088 123826176 25851 268435456000 4194304 4587354 140735275698096 18446744073709551615 4236580 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/4978/statm: 30231 25851 328 96 0 26793 0
[pid=4979] ppid=4977 vsize=178256 CPUtime=12.69 cores=1
/proc/4979/stat : 4979 (clasp) R 4977 4977 4951 0 -1 4202496 47308 0 0 0 1257 12 0 0 25 0 1 0 617446088 182534144 41658 268435456000 4194304 4785740 140736924365072 18446744073709551615 4304033 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4979/statm: 44564 41658 338 145 0 41609 0
[pid=4980] ppid=4977 vsize=509684 CPUtime=12.68 cores=2
/proc/4980/stat : 4980 (TNM) R 4977 4977 4951 0 -1 4202496 23080 0 0 0 1263 5 0 0 25 0 1 0 617446089 521916416 22978 268435456000 134512640 134534564 4290360448 18446744073709551615 134530632 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/4980/statm: 127421 22978 95 6 0 126086 0
[pid=4981] ppid=4977 vsize=104608 CPUtime=12.69 cores=3
/proc/4981/stat : 4981 (march_hi) R 4977 4977 4951 0 -1 4194304 186330 0 0 0 1226 43 0 0 25 0 1 0 617446089 107118592 18103 268435456000 4194304 4331988 140736480980336 18446744073709551615 258650611670 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/4981/statm: 26152 18103 127 34 0 24594 0
[pid=4982] ppid=4977 vsize=1065064 CPUtime=169.04 cores=4-31
/proc/4982/stat : 4982 (plingeling) S 4977 4977 4951 0 -1 4202496 315438 1239 0 0 16814 90 0 0 19 0 29 0 617446089 1090625536 149966 268435456000 4194304 4356188 140734849797200 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 12 0 0 0
/proc/4982/statm: 266266 149966 159 40 0 264810 0
[pid=4982/tid=4989] ppid=4977 vsize=1065064 CPUtime=5.8 cores=4-31
/proc/4982/task/4989/stat : 4989 (plingeling) R 4977 4977 4951 0 -1 4202560 6341 1239 0 0 579 1 0 0 25 0 29 0 617446772 1090625536 149966 268435456000 4194304 4356188 140734849797200 18446744073709551615 4281793 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=4982/tid=4990] ppid=4977 vsize=1065064 CPUtime=5.82 cores=4-31
/proc/4982/task/4990/stat : 4990 (plingeling) R 4977 4977 4951 0 -1 4202560 3950 1239 0 0 580 2 0 0 25 0 29 0 617446772 1090625536 149966 268435456000 4194304 4356188 140734849797200 18446744073709551615 4211883 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=4982/tid=4991] ppid=4977 vsize=1065064 CPUtime=5.8 cores=4-31
/proc/4982/task/4991/stat : 4991 (plingeling) R 4977 4977 4951 0 -1 4202560 5756 1239 0 0 578 2 0 0 25 0 29 0 617446772 1090625536 149966 268435456000 4194304 4356188 140734849797200 18446744073709551615 4210557 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=4982/tid=4992] ppid=4977 vsize=1065064 CPUtime=5.81 cores=4-31
/proc/4982/task/4992/stat : 4992 (plingeling) R 4977 4977 4951 0 -1 4202560 3478 1239 0 0 580 1 0 0 25 0 29 0 617446772 1090625536 149966 268435456000 4194304 4356188 140734849797200 18446744073709551615 4334501 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4982/tid=4993] ppid=4977 vsize=1065064 CPUtime=5.81 cores=4-31
/proc/4982/task/4993/stat : 4993 (plingeling) R 4977 4977 4951 0 -1 4202560 2884 1239 0 0 581 0 0 0 25 0 29 0 617446772 1090625536 149966 268435456000 4194304 4356188 140734849797200 18446744073709551615 4338451 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4982/tid=4994] ppid=4977 vsize=1065064 CPUtime=5.8 cores=4-31
/proc/4982/task/4994/stat : 4994 (plingeling) R 4977 4977 4951 0 -1 4202560 8271 1239 0 0 577 3 0 0 25 0 29 0 617446772 1090625536 149966 268435456000 4194304 4356188 140734849797200 18446744073709551615 4281672 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=4982/tid=4995] ppid=4977 vsize=1065064 CPUtime=5.8 cores=4-31
/proc/4982/task/4995/stat : 4995 (plingeling) R 4977 4977 4951 0 -1 4202560 7360 1239 0 0 578 2 0 0 25 0 29 0 617446772 1090625536 149966 268435456000 4194304 4356188 140734849797200 18446744073709551615 4281672 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=4982/tid=4996] ppid=4977 vsize=1065064 CPUtime=5.78 cores=4-31
/proc/4982/task/4996/stat : 4996 (plingeling) R 4977 4977 4951 0 -1 4202560 2499 1239 0 0 578 0 0 0 25 0 29 0 617446772 1090625536 149966 268435456000 4194304 4356188 140734849797200 18446744073709551615 4283929 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=4982/tid=4997] ppid=4977 vsize=1065064 CPUtime=5.8 cores=4-31
/proc/4982/task/4997/stat : 4997 (plingeling) R 4977 4977 4951 0 -1 4202560 5258 1239 0 0 579 1 0 0 25 0 29 0 617446772 1090625536 149966 268435456000 4194304 4356188 140734849797200 18446744073709551615 4270551 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4982/tid=4998] ppid=4977 vsize=1065064 CPUtime=5.79 cores=4-31
/proc/4982/task/4998/stat : 4998 (plingeling) R 4977 4977 4951 0 -1 4202560 4971 1239 0 0 577 2 0 0 25 0 29 0 617446773 1090625536 149966 268435456000 4194304 4356188 140734849797200 18446744073709551615 4334467 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=4982/tid=4999] ppid=4977 vsize=1065064 CPUtime=5.79 cores=4-31
/proc/4982/task/4999/stat : 4999 (plingeling) R 4977 4977 4951 0 -1 4202560 5045 1239 0 0 577 2 0 0 25 0 29 0 617446773 1090625536 149966 268435456000 4194304 4356188 140734849797200 18446744073709551615 4281760 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=4982/tid=5000] ppid=4977 vsize=1065064 CPUtime=5.78 cores=4-31
/proc/4982/task/5000/stat : 5000 (plingeling) R 4977 4977 4951 0 -1 4202560 5206 1239 0 0 577 1 0 0 25 0 29 0 617446773 1090625536 149966 268435456000 4194304 4356188 140734849797200 18446744073709551615 4234076 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4982/tid=5001] ppid=4977 vsize=1065064 CPUtime=5.79 cores=4-31
/proc/4982/task/5001/stat : 5001 (plingeling) R 4977 4977 4951 0 -1 4202560 5863 1239 0 0 578 1 0 0 25 0 29 0 617446773 1090625536 149966 268435456000 4194304 4356188 140734849797200 18446744073709551615 4334501 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=4982/tid=5002] ppid=4977 vsize=1065064 CPUtime=5.79 cores=4-31
/proc/4982/task/5002/stat : 5002 (plingeling) R 4977 4977 4951 0 -1 4202560 5159 1239 0 0 578 1 0 0 25 0 29 0 617446773 1090625536 149966 268435456000 4194304 4356188 140734849797200 18446744073709551615 4281643 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=4982/tid=5003] ppid=4977 vsize=1065064 CPUtime=5.77 cores=4-31
/proc/4982/task/5003/stat : 5003 (plingeling) R 4977 4977 4951 0 -1 4202560 6645 1239 0 0 575 2 0 0 25 0 29 0 617446773 1090625536 149966 268435456000 4194304 4356188 140734849797200 18446744073709551615 4331050 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=4982/tid=5004] ppid=4977 vsize=1065064 CPUtime=5.78 cores=4-31
/proc/4982/task/5004/stat : 5004 (plingeling) R 4977 4977 4951 0 -1 4202560 4694 1239 0 0 577 1 0 0 25 0 29 0 617446774 1090625536 149966 268435456000 4194304 4356188 140734849797200 18446744073709551615 4283908 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=4982/tid=5005] ppid=4977 vsize=1065064 CPUtime=5.8 cores=4-31
/proc/4982/task/5005/stat : 5005 (plingeling) R 4977 4977 4951 0 -1 4202560 5384 1239 0 0 579 1 0 0 25 0 29 0 617446774 1090625536 149966 268435456000 4194304 4356188 140734849797200 18446744073709551615 258650649892 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=4982/tid=5006] ppid=4977 vsize=1065064 CPUtime=5.79 cores=4-31
/proc/4982/task/5006/stat : 5006 (plingeling) R 4977 4977 4951 0 -1 4202560 2382 1239 0 0 578 1 0 0 25 0 29 0 617446774 1090625536 149966 268435456000 4194304 4356188 140734849797200 18446744073709551615 4210544 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=4982/tid=5007] ppid=4977 vsize=1065064 CPUtime=5.77 cores=4-31
/proc/4982/task/5007/stat : 5007 (plingeling) R 4977 4977 4951 0 -1 4202560 1964 1239 0 0 577 0 0 0 25 0 29 0 617446774 1090625536 149966 268435456000 4194304 4356188 140734849797200 18446744073709551615 4234339 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=4982/tid=5008] ppid=4977 vsize=1065064 CPUtime=5.77 cores=4-31
/proc/4982/task/5008/stat : 5008 (plingeling) R 4977 4977 4951 0 -1 4202560 6352 1239 0 0 575 2 0 0 25 0 29 0 617446774 1090625536 149966 268435456000 4194304 4356188 140734849797200 18446744073709551615 4281672 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=4982/tid=5009] ppid=4977 vsize=1065064 CPUtime=5.8 cores=4-31
/proc/4982/task/5009/stat : 5009 (plingeling) R 4977 4977 4951 0 -1 4202560 3049 1239 0 0 578 2 0 0 25 0 29 0 617446774 1090625536 149966 268435456000 4194304 4356188 140734849797200 18446744073709551615 4211703 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=4982/tid=5010] ppid=4977 vsize=1065064 CPUtime=5.79 cores=4-31

################
# More data... #
################

[pid=4980] ppid=4977 vsize=509684 CPUtime=4961.37 cores=2
/proc/4980/stat : 4980 (TNM) R 4977 4977 4951 0 -1 4202496 23080 0 0 0 496132 5 0 0 25 0 1 0 617446089 521916416 22978 268435456000 134512640 134534564 4290360448 18446744073709551615 134530151 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/4980/statm: 127421 22978 95 6 0 126086 0
[pid=4981] ppid=4977 vsize=333408 CPUtime=4961.34 cores=3
/proc/4981/stat : 4981 (march_hi) R 4977 4977 4951 0 -1 4202496 272927 0 0 0 496066 68 0 0 25 0 1 0 617446089 341409792 42602 268435456000 4194304 4331988 140736480980336 18446744073709551615 4210418 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/4981/statm: 83352 42602 181 34 0 81794 0
[pid=4982] ppid=4977 vsize=10160824 CPUtime=138563 cores=4-31
/proc/4982/stat : 4982 (plingeling) S 4977 4977 4951 0 -1 4202496 29543265 1239 0 0 13837431 18845 0 0 19 0 29 0 617446089 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 12 0 0 0
/proc/4982/statm: 2540206 2416738 159 40 0 2538750 0
[pid=4982/tid=4989] ppid=4977 vsize=10160824 CPUtime=4942.88 cores=4-31
/proc/4982/task/4989/stat : 4989 (plingeling) R 4977 4977 4951 0 -1 4202560 1043744 1239 0 0 493610 678 0 0 25 0 29 0 617446772 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4284244 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=4982/tid=4990] ppid=4977 vsize=10160824 CPUtime=4953.53 cores=4-31
/proc/4982/task/4990/stat : 4990 (plingeling) R 4977 4977 4951 0 -1 4202560 1007558 1239 0 0 494809 544 0 0 25 0 29 0 617446772 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4244324 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=4982/tid=4991] ppid=4977 vsize=10160824 CPUtime=4953.59 cores=4-31
/proc/4982/task/4991/stat : 4991 (plingeling) R 4977 4977 4951 0 -1 4202560 988635 1239 0 0 494740 619 0 0 25 0 29 0 617446772 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4281624 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=4982/tid=4992] ppid=4977 vsize=10160824 CPUtime=4953.52 cores=4-31
/proc/4982/task/4992/stat : 4992 (plingeling) R 4977 4977 4951 0 -1 4202560 987541 1239 0 0 494784 568 0 0 25 0 29 0 617446772 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4238570 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4982/tid=4993] ppid=4977 vsize=10160824 CPUtime=4953.51 cores=4-31
/proc/4982/task/4993/stat : 4993 (plingeling) R 4977 4977 4951 0 -1 4202560 966251 1239 0 0 494874 477 0 0 25 0 29 0 617446772 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4279490 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4982/tid=4994] ppid=4977 vsize=10160824 CPUtime=4953.62 cores=4-31
/proc/4982/task/4994/stat : 4994 (plingeling) R 4977 4977 4951 0 -1 4202560 1065700 1239 0 0 494772 590 0 0 25 0 29 0 617446772 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=4982/tid=4995] ppid=4977 vsize=10160824 CPUtime=4952.99 cores=4-31
/proc/4982/task/4995/stat : 4995 (plingeling) R 4977 4977 4951 0 -1 4202560 856043 1239 0 0 494675 624 0 0 25 0 29 0 617446772 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4281771 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=4982/tid=4996] ppid=4977 vsize=10160824 CPUtime=4953.66 cores=4-31
/proc/4982/task/4996/stat : 4996 (plingeling) R 4977 4977 4951 0 -1 4202560 1062034 1239 0 0 494120 1246 0 0 25 0 29 0 617446772 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=4982/tid=4997] ppid=4977 vsize=10160824 CPUtime=4943.68 cores=4-31
/proc/4982/task/4997/stat : 4997 (plingeling) R 4977 4977 4951 0 -1 4202560 901425 1239 0 0 493884 484 0 0 25 0 29 0 617446772 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4284253 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=4982/tid=4998] ppid=4977 vsize=10160824 CPUtime=4953.23 cores=4-31
/proc/4982/task/4998/stat : 4998 (plingeling) R 4977 4977 4951 0 -1 4202560 969995 1239 0 0 494804 519 0 0 25 0 29 0 617446773 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4284253 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=4982/tid=4999] ppid=4977 vsize=10160824 CPUtime=4953.83 cores=4-31
/proc/4982/task/4999/stat : 4999 (plingeling) R 4977 4977 4951 0 -1 4202560 1094508 1239 0 0 494831 552 0 0 25 0 29 0 617446773 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4281672 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=4982/tid=5000] ppid=4977 vsize=10160824 CPUtime=4953.61 cores=4-31
/proc/4982/task/5000/stat : 5000 (plingeling) R 4977 4977 4951 0 -1 4202560 1043978 1239 0 0 494708 653 0 0 25 0 29 0 617446773 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4210834 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4982/tid=5001] ppid=4977 vsize=10160824 CPUtime=4941.14 cores=4-31
/proc/4982/task/5001/stat : 5001 (plingeling) R 4977 4977 4951 0 -1 4202560 1065583 1239 0 0 493494 620 0 0 25 0 29 0 617446773 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4238570 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=4982/tid=5002] ppid=4977 vsize=10160824 CPUtime=4932.51 cores=4-31
/proc/4982/task/5002/stat : 5002 (plingeling) R 4977 4977 4951 0 -1 4202560 1007684 1239 0 0 492637 614 0 0 25 0 29 0 617446773 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4332020 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=4982/tid=5003] ppid=4977 vsize=10160824 CPUtime=4953.37 cores=4-31
/proc/4982/task/5003/stat : 5003 (plingeling) R 4977 4977 4951 0 -1 4202560 1131879 1239 0 0 494659 678 0 0 25 0 29 0 617446773 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4331340 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=4982/tid=5004] ppid=4977 vsize=10160824 CPUtime=4939.12 cores=4-31
/proc/4982/task/5004/stat : 5004 (plingeling) R 4977 4977 4951 0 -1 4202560 882947 1239 0 0 492739 1173 0 0 25 0 29 0 617446774 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4210834 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=4982/tid=5005] ppid=4977 vsize=10160824 CPUtime=4953.66 cores=4-31
/proc/4982/task/5005/stat : 5005 (plingeling) R 4977 4977 4951 0 -1 4202560 1273913 1239 0 0 494729 637 0 0 25 0 29 0 617446774 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4284190 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=4982/tid=5006] ppid=4977 vsize=10160824 CPUtime=4953.17 cores=4-31
/proc/4982/task/5006/stat : 5006 (plingeling) R 4977 4977 4951 0 -1 4202560 984274 1239 0 0 494734 583 0 0 25 0 29 0 617446774 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4210575 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=4982/tid=5007] ppid=4977 vsize=10160824 CPUtime=4948.66 cores=4-31
/proc/4982/task/5007/stat : 5007 (plingeling) R 4977 4977 4951 0 -1 4202560 1082627 1239 0 0 494246 620 0 0 25 0 29 0 617446774 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4284178 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4982/tid=5008] ppid=4977 vsize=10160824 CPUtime=4952.11 cores=4-31
/proc/4982/task/5008/stat : 5008 (plingeling) R 4977 4977 4951 0 -1 4202560 1110625 1239 0 0 494503 708 0 0 25 0 29 0 617446774 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4281672 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=4982/tid=5009] ppid=4977 vsize=10160824 CPUtime=4951.81 cores=4-31
/proc/4982/task/5009/stat : 5009 (plingeling) R 4977 4977 4951 0 -1 4202560 1208423 1239 0 0 494453 728 0 0 25 0 29 0 617446774 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=4982/tid=5010] ppid=4977 vsize=10160824 CPUtime=4934.06 cores=4-31
/proc/4982/task/5010/stat : 5010 (plingeling) R 4977 4977 4951 0 -1 4202560 1240603 1239 0 0 492540 866 0 0 25 0 29 0 617446774 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4281692 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=4982/tid=5011] ppid=4977 vsize=10160824 CPUtime=4953.94 cores=4-31
/proc/4982/task/5011/stat : 5011 (plingeling) R 4977 4977 4951 0 -1 4202560 1083587 1239 0 0 494669 725 0 0 25 0 29 0 617446774 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4234306 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=4982/tid=5012] ppid=4977 vsize=10160824 CPUtime=4952.99 cores=4-31
/proc/4982/task/5012/stat : 5012 (plingeling) R 4977 4977 4951 0 -1 4202560 1003459 1239 0 0 494141 1158 0 0 25 0 29 0 617446775 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=4982/tid=5013] ppid=4977 vsize=10160824 CPUtime=4947.74 cores=4-31
/proc/4982/task/5013/stat : 5013 (plingeling) R 4977 4977 4951 0 -1 4202560 1096996 1239 0 0 494243 531 0 0 25 0 29 0 617446775 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4234009 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=4982/tid=5014] ppid=4977 vsize=10160824 CPUtime=4927.53 cores=4-31
/proc/4982/task/5014/stat : 5014 (plingeling) R 4977 4977 4951 0 -1 4202560 1020504 1239 0 0 492286 467 0 0 25 0 29 0 617446775 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4281760 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=4982/tid=5015] ppid=4977 vsize=10160824 CPUtime=4939.84 cores=4-31
/proc/4982/task/5015/stat : 5015 (plingeling) R 4977 4977 4951 0 -1 4202560 1184624 1239 0 0 493397 587 0 0 25 0 29 0 617446775 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4281672 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=4982/tid=5016] ppid=4977 vsize=10160824 CPUtime=4952.38 cores=4-31
/proc/4982/task/5016/stat : 5016 (plingeling) R 4977 4977 4951 0 -1 4202560 1001798 1239 0 0 494692 546 0 0 25 0 29 0 617446775 10404683776 2416738 268435456000 4194304 4356188 140734849797200 18446744073709551615 4283920 0 0 4096 17442 0 0 0 -1 12 0 0 0
Current children cumulated CPU time (s) 158407
Current children cumulated vsize (KiB) 16672156



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5012.2 s]
/proc/loadavg: 32.13 32.13 32.09 33/447 5162
/proc/meminfo: memFree=241847776/264257624 swapFree=67111528/67111528
[pid=4977] ppid=4975 vsize=18324 CPUtime=0.07 cores=0-31
/proc/4977/stat : 4977 (ppfolio) S 4975 4977 4951 0 -1 4202496 515 0 0 0 1 6 0 0 15 0 1 0 617446088 18763776 386 268435456000 4194304 4287625 140736406997040 18446744073709551615 258650975999 0 0 4096 65536 9197450367330156543 0 0 17 12 0 0 0
/proc/4977/statm: 4581 386 337 23 0 91 0
[pid=4978] ppid=4977 vsize=2333484 CPUtime=5011.04 cores=0
/proc/4978/stat : 4978 (cryptominisat) R 4977 4977 4951 0 -1 4202496 2436042 0 0 0 500230 874 0 0 25 0 1 0 617446088 2389487616 123166 268435456000 4194304 4587354 140735275698096 18446744073709551615 4238611 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/4978/statm: 583371 123166 338 96 0 579933 0
[pid=4979] ppid=4977 vsize=3381956 CPUtime=5010.12 cores=1
/proc/4979/stat : 4979 (clasp) R 4977 4977 4951 0 -1 4202496 845152 0 0 0 500750 262 0 0 25 0 1 0 617446088 3463122944 839502 268435456000 4194304 4785740 140736924365072 18446744073709551615 4278573 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4979/statm: 845489 839502 338 145 0 842534 0
[pid=4980] ppid=4977 vsize=509684 CPUtime=5011.25 cores=2
/proc/4980/stat : 4980 (TNM) R 4977 4977 4951 0 -1 4202496 23080 0 0 0 501120 5 0 0 25 0 1 0 617446089 521916416 22978 268435456000 134512640 134534564 4290360448 18446744073709551615 134528708 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/4980/statm: 127421 22978 95 6 0 126086 0
[pid=4981] ppid=4977 vsize=333540 CPUtime=5011.24 cores=3
/proc/4981/stat : 4981 (march_hi) R 4977 4977 4951 0 -1 4202496 273050 0 0 0 501055 69 0 0 25 0 1 0 617446089 341544960 42725 268435456000 4194304 4331988 140736480980336 18446744073709551615 4228288 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/4981/statm: 83385 42725 181 34 0 81827 0
[pid=4982] ppid=4977 vsize=10749656 CPUtime=139957 cores=4-31
/proc/4982/stat : 4982 (plingeling) S 4977 4977 4951 0 -1 4202496 29868989 1239 0 0 13976578 19120 0 0 19 0 29 0 617446089 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 12 0 0 0
/proc/4982/statm: 2687414 2563946 159 40 0 2685958 0
[pid=4982/tid=4989] ppid=4977 vsize=10749656 CPUtime=4992.77 cores=4-31
/proc/4982/task/4989/stat : 4989 (plingeling) R 4977 4977 4951 0 -1 4202560 1068320 1239 0 0 498586 691 0 0 25 0 29 0 617446772 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4281652 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=4982/tid=4990] ppid=4977 vsize=10749656 CPUtime=5003.42 cores=4-31
/proc/4982/task/4990/stat : 4990 (plingeling) R 4977 4977 4951 0 -1 4202560 1012678 1239 0 0 499797 545 0 0 25 0 29 0 617446772 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4212151 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=4982/tid=4991] ppid=4977 vsize=10749656 CPUtime=5003.39 cores=4-31
/proc/4982/task/4991/stat : 4991 (plingeling) R 4977 4977 4951 0 -1 4202560 1012187 1239 0 0 499706 633 0 0 25 0 29 0 617446772 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4281793 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=4982/tid=4992] ppid=4977 vsize=10749656 CPUtime=5003.38 cores=4-31
/proc/4982/task/4992/stat : 4992 (plingeling) R 4977 4977 4951 0 -1 4202560 1003925 1239 0 0 499767 571 0 0 25 0 29 0 617446772 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4332020 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=4982/tid=4993] ppid=4977 vsize=10749656 CPUtime=5003.4 cores=4-31
/proc/4982/task/4993/stat : 4993 (plingeling) R 4977 4977 4951 0 -1 4202560 970347 1239 0 0 499862 478 0 0 25 0 29 0 617446772 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4281672 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4982/tid=4994] ppid=4977 vsize=10749656 CPUtime=5003.51 cores=4-31
/proc/4982/task/4994/stat : 4994 (plingeling) R 4977 4977 4951 0 -1 4202560 1075940 1239 0 0 499759 592 0 0 25 0 29 0 617446772 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4211010 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=4982/tid=4995] ppid=4977 vsize=10749656 CPUtime=5002.88 cores=4-31
/proc/4982/task/4995/stat : 4995 (plingeling) R 4977 4977 4951 0 -1 4202560 874476 1239 0 0 499661 627 0 0 25 0 29 0 617446772 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4331074 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=4982/tid=4996] ppid=4977 vsize=10749656 CPUtime=5003.55 cores=4-31
/proc/4982/task/4996/stat : 4996 (plingeling) R 4977 4977 4951 0 -1 4202560 1062034 1239 0 0 499109 1246 0 0 25 0 29 0 617446772 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=4982/tid=4997] ppid=4977 vsize=10749656 CPUtime=4993.49 cores=4-31
/proc/4982/task/4997/stat : 4997 (plingeling) R 4977 4977 4951 0 -1 4202560 921905 1239 0 0 498861 488 0 0 25 0 29 0 617446772 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4279742 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=4982/tid=4998] ppid=4977 vsize=10749656 CPUtime=5003.13 cores=4-31
/proc/4982/task/4998/stat : 4998 (plingeling) R 4977 4977 4951 0 -1 4202560 969995 1239 0 0 499793 520 0 0 25 0 29 0 617446773 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4281672 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=4982/tid=4999] ppid=4977 vsize=10749656 CPUtime=5003.72 cores=4-31
/proc/4982/task/4999/stat : 4999 (plingeling) R 4977 4977 4951 0 -1 4202560 1119084 1239 0 0 499796 576 0 0 25 0 29 0 617446773 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4234244 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=4982/tid=5000] ppid=4977 vsize=10749656 CPUtime=5003.5 cores=4-31
/proc/4982/task/5000/stat : 5000 (plingeling) R 4977 4977 4951 0 -1 4202560 1043978 1239 0 0 499697 653 0 0 25 0 29 0 617446773 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4284197 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4982/tid=5001] ppid=4977 vsize=10749656 CPUtime=4990.49 cores=4-31
/proc/4982/task/5001/stat : 5001 (plingeling) R 4977 4977 4951 0 -1 4202560 1073866 1239 0 0 498427 622 0 0 25 0 29 0 617446773 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4281798 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4982/tid=5002] ppid=4977 vsize=10749656 CPUtime=4982.4 cores=4-31
/proc/4982/task/5002/stat : 5002 (plingeling) R 4977 4977 4951 0 -1 4202560 1028164 1239 0 0 497606 634 0 0 25 0 29 0 617446773 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4210544 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=4982/tid=5003] ppid=4977 vsize=10749656 CPUtime=5003.26 cores=4-31
/proc/4982/task/5003/stat : 5003 (plingeling) R 4977 4977 4951 0 -1 4202560 1144167 1239 0 0 499645 681 0 0 25 0 29 0 617446773 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4283983 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=4982/tid=5004] ppid=4977 vsize=10749656 CPUtime=4989.02 cores=4-31
/proc/4982/task/5004/stat : 5004 (plingeling) R 4977 4977 4951 0 -1 4202560 891139 1239 0 0 497708 1194 0 0 25 0 29 0 617446774 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=4982/tid=5005] ppid=4977 vsize=10749656 CPUtime=5003.56 cores=4-31
/proc/4982/task/5005/stat : 5005 (plingeling) R 4977 4977 4951 0 -1 4202560 1273913 1239 0 0 499719 637 0 0 25 0 29 0 617446774 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4281793 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=4982/tid=5006] ppid=4977 vsize=10749656 CPUtime=5002.37 cores=4-31
/proc/4982/task/5006/stat : 5006 (plingeling) R 4977 4977 4951 0 -1 4202560 1000658 1239 0 0 499650 587 0 0 25 0 29 0 617446774 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4281652 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=4982/tid=5007] ppid=4977 vsize=10749656 CPUtime=4998.56 cores=4-31
/proc/4982/task/5007/stat : 5007 (plingeling) R 4977 4977 4951 0 -1 4202560 1082627 1239 0 0 499236 620 0 0 25 0 29 0 617446774 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4283938 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4982/tid=5008] ppid=4977 vsize=10749656 CPUtime=5001.98 cores=4-31
/proc/4982/task/5008/stat : 5008 (plingeling) R 4977 4977 4951 0 -1 4202560 1122913 1239 0 0 499460 738 0 0 25 0 29 0 617446774 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4225795 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=4982/tid=5009] ppid=4977 vsize=10749656 CPUtime=5001.7 cores=4-31
/proc/4982/task/5009/stat : 5009 (plingeling) R 4977 4977 4951 0 -1 4202560 1208423 1239 0 0 499442 728 0 0 25 0 29 0 617446774 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=4982/tid=5010] ppid=4977 vsize=10749656 CPUtime=4983.95 cores=4-31
/proc/4982/task/5010/stat : 5010 (plingeling) R 4977 4977 4951 0 -1 4202560 1240603 1239 0 0 497529 866 0 0 25 0 29 0 617446774 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4219786 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=4982/tid=5011] ppid=4977 vsize=10749656 CPUtime=5003.83 cores=4-31
/proc/4982/task/5011/stat : 5011 (plingeling) R 4977 4977 4951 0 -1 4202560 1083587 1239 0 0 499658 725 0 0 25 0 29 0 617446774 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4210554 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=4982/tid=5012] ppid=4977 vsize=10749656 CPUtime=5001.97 cores=4-31
/proc/4982/task/5012/stat : 5012 (plingeling) R 4977 4977 4951 0 -1 4202560 1054659 1239 0 0 498918 1279 0 0 25 0 29 0 617446775 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4336036 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=4982/tid=5013] ppid=4977 vsize=10749656 CPUtime=4997.44 cores=4-31
/proc/4982/task/5013/stat : 5013 (plingeling) R 4977 4977 4951 0 -1 4202560 1101092 1239 0 0 499211 533 0 0 25 0 29 0 617446775 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4281502 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=4982/tid=5014] ppid=4977 vsize=10749656 CPUtime=4977.43 cores=4-31
/proc/4982/task/5014/stat : 5014 (plingeling) R 4977 4977 4951 0 -1 4202560 1028696 1239 0 0 497274 469 0 0 25 0 29 0 617446775 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4281782 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=4982/tid=5015] ppid=4977 vsize=10749656 CPUtime=4989.71 cores=4-31
/proc/4982/task/5015/stat : 5015 (plingeling) R 4977 4977 4951 0 -1 4202560 1196912 1239 0 0 498382 589 0 0 25 0 29 0 617446775 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4234297 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=4982/tid=5016] ppid=4977 vsize=10749656 CPUtime=5002.27 cores=4-31
/proc/4982/task/5016/stat : 5016 (plingeling) R 4977 4977 4951 0 -1 4202560 1026374 1239 0 0 499677 550 0 0 25 0 29 0 617446775 11007647744 2563946 268435456000 4194304 4356188 140734849797200 18446744073709551615 4241979 0 0 4096 17442 0 0 0 -1 12 0 0 0
Current children cumulated CPU time (s) 160001
Current children cumulated vsize (KiB) 17326644

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 4977 and gives
#  childrusage.ru_utime.tv_sec=5011
#  childrusage.ru_utime.tv_usec=236176
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=132979
# CPU time returned by wait4() is 5011.37
# while last known CPU time is 160001
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5012.24
CPU time (s): 160001
CPU user time (s): 159797
CPU system time (s): 203.36
CPU usage (%): 3192.2
Max. virtual memory (cumulated for all children) (KiB): 18892284

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 5011.24
system time used= 0.132979
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 23597
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 7563
involuntary context switches= 17122

runsolver used 28.2437 second user time and 113.557 second system time

The end

Launcher Data 

Begin job on node200 at 2011-05-10 22:33:17
IDJOB=3347363
IDBENCH=82372
IDSOLVER=1808
FILE ID=node200/3347363-1305059596
RUNJOBID= node200-1305059596-4965
PBS_JOBID= 13282047
Free space on /tmp= 62040 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/application/rintanen/SATPlanning/openstacks-sequencedstrips-nonadl-nonnegated-os-sequencedstrips-p30_3.035-NOTKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3347363-1305059596/watcher-3347363-1305059596 -o /tmp/evaluation-result-3347363-1305059596/solver-3347363-1305059596 -C 160000 -W 5100 -M 266338304  ppfolio HOME/instance-3347363-1305059596.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 266338304 MiB
NBCORE= 32

MD5SUM BENCH= c4319228342e10028de028a9b67c3404
RANDOM SEED=132935324

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.98
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.23
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      256237680 kB
Buffers:        382212 kB
Cached:        7288240 kB
SwapCached:          0 kB
Active:        6803740 kB
Inactive:       935512 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      256237680 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           21784 kB
Writeback:           0 kB
AnonPages:       69248 kB
Mapped:          13484 kB
Slab:           195736 kB
PageTables:       4336 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   193928 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 62028 MiB
End job on node200 at 2011-05-10 23:56:51