Trace number 3347731

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 160002 5044.14

General information on the benchmark

NameSAT-Race-2010-CNF/hardware-verification/
velev/11pipe_11_ooo.cnf
MD5SUMf603f11fbfe855df381282b7bce010ea
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark46232.8
Satisfiable
(Un)Satisfiability was proved
Number of variables88289
Number of clauses4187694
Sum of the clauses size12410354
Maximum clause length341
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 23770000
Number of clauses of size 3347602
Number of clauses of size 41622
Number of clauses of size 51351
Number of clauses of size over 567118

Solver Data

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 32 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

Verifier Data

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3347731-1306157750/watcher-3347731-1306157750 -o /tmp/evaluation-result-3347731-1306157750/solver-3347731-1306157750 -C 160000 -W 5100 -M 266338304 ppfolio HOME/instance-3347731-1306157750.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 272730423296 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 272730474496 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 29.55 31.50 30.63 1/413 3374
/proc/meminfo: memFree=255151620/264257624 swapFree=67111528/67111528
[pid=3374] ppid=3372 vsize=18324 CPUtime=0 cores=0-31
/proc/3374/stat : 3374 (ppfolio) S 3372 3374 3315 0 -1 4202496 511 0 0 0 0 0 0 0 23 0 1 0 727261750 18763776 385 268435456000 4194304 4287625 140735800438592 18446744073709551615 258650975999 0 0 4096 65536 9170076650967138303 0 0 17 16 0 0 0
/proc/3374/statm: 4581 385 337 23 0 91 0

[startup+0.047621 s]
/proc/loadavg: 29.55 31.50 30.63 1/413 3374
/proc/meminfo: memFree=255151620/264257624 swapFree=67111528/67111528
[pid=3374] ppid=3372 vsize=18324 CPUtime=0 cores=0-31
/proc/3374/stat : 3374 (ppfolio) S 3372 3374 3315 0 -1 4202496 511 0 0 0 0 0 0 0 23 0 1 0 727261750 18763776 385 268435456000 4194304 4287625 140735800438592 18446744073709551615 258650975999 0 0 4096 65536 9170076650967138303 0 0 17 16 0 0 0
/proc/3374/statm: 4581 385 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100608 s]
/proc/loadavg: 29.55 31.50 30.63 1/413 3374
/proc/meminfo: memFree=255151620/264257624 swapFree=67111528/67111528
[pid=3374] ppid=3372 vsize=18324 CPUtime=0 cores=0-31
/proc/3374/stat : 3374 (ppfolio) S 3372 3374 3315 0 -1 4202496 511 0 0 0 0 0 0 0 23 0 1 0 727261750 18763776 385 268435456000 4194304 4287625 140735800438592 18446744073709551615 258650975999 0 0 4096 65536 9170076650967138303 0 0 17 16 0 0 0
/proc/3374/statm: 4581 385 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300648 s]
/proc/loadavg: 29.55 31.50 30.63 1/413 3374
/proc/meminfo: memFree=255151620/264257624 swapFree=67111528/67111528
[pid=3374] ppid=3372 vsize=18324 CPUtime=0 cores=0-31
/proc/3374/stat : 3374 (ppfolio) S 3372 3374 3315 0 -1 4202496 511 0 0 0 0 0 0 0 23 0 1 0 727261750 18763776 385 268435456000 4194304 4287625 140735800438592 18446744073709551615 258650975999 0 0 4096 65536 9170076650967138303 0 0 17 16 0 0 0
/proc/3374/statm: 4581 385 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700642 s]
/proc/loadavg: 29.55 31.50 30.63 1/413 3374
/proc/meminfo: memFree=255151620/264257624 swapFree=67111528/67111528
[pid=3374] ppid=3372 vsize=18324 CPUtime=0 cores=0-31
/proc/3374/stat : 3374 (ppfolio) S 3372 3374 3315 0 -1 4202496 511 0 0 0 0 0 0 0 23 0 1 0 727261750 18763776 385 268435456000 4194304 4287625 140735800438592 18446744073709551615 258650975999 0 0 4096 65536 9170076650967138303 0 0 17 16 0 0 0
/proc/3374/statm: 4581 385 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50063 s]
/proc/loadavg: 27.58 31.06 30.49 6/419 3385
/proc/meminfo: memFree=254830920/264257624 swapFree=67111528/67111528
[pid=3374] ppid=3372 vsize=18324 CPUtime=0 cores=0-31
/proc/3374/stat : 3374 (ppfolio) S 3372 3374 3315 0 -1 4202496 511 0 0 0 0 0 0 0 23 0 1 0 727261750 18763776 385 268435456000 4194304 4287625 140735800438592 18446744073709551615 258650975999 0 0 4096 65536 9170076650967138303 0 0 17 16 0 0 0
/proc/3374/statm: 4581 385 337 23 0 91 0
[pid=3375] ppid=3374 vsize=254064 CPUtime=1.48 cores=0
/proc/3375/stat : 3375 (cryptominisat) R 3374 3374 3315 0 -1 4202496 49940 0 0 0 132 16 0 0 20 0 1 0 727261750 260161536 49708 268435456000 4194304 4587354 140733290898224 18446744073709551615 4440913 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/3375/statm: 63516 49708 285 96 0 60078 0
[pid=3376] ppid=3374 vsize=77412 CPUtime=1.48 cores=1
/proc/3376/stat : 3376 (clasp) R 3374 3374 3315 0 -1 4202496 17184 0 0 0 137 11 0 0 20 0 1 0 727261750 79269888 16428 268435456000 4194304 4785740 140735199264048 18446744073709551615 258650616632 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/3376/statm: 19353 16428 306 145 0 16398 0
[pid=3377] ppid=3374 vsize=471144 CPUtime=1.48 cores=2
/proc/3377/stat : 3377 (TNM) R 3374 3374 3315 0 -1 4202496 7946 0 0 0 144 4 0 0 20 0 1 0 727261750 482451456 7845 268435456000 134512640 134534564 4292630352 18446744073709551615 134520971 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/3377/statm: 117786 7845 90 6 0 116451 0
[pid=3378] ppid=3374 vsize=119748 CPUtime=1.48 cores=3
/proc/3378/stat : 3378 (march_hi) R 3374 3374 3315 0 -1 4194304 25952 0 0 0 138 10 0 0 20 0 1 0 727261750 122621952 25848 268435456000 4194304 4331988 140733561838752 18446744073709551615 258650616632 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/3378/statm: 29937 25849 112 34 0 28379 0
[pid=3379] ppid=3374 vsize=108520 CPUtime=1.49 cores=4-31
/proc/3379/stat : 3379 (plingeling) R 3374 3374 3315 0 -1 4202496 25977 1239 0 0 142 6 0 1 21 0 1 0 727261750 111124480 25765 268435456000 4194304 4356188 140734709006800 18446744073709551615 4213270 0 0 4096 17442 0 0 0 17 10 0 0 0
/proc/3379/statm: 27130 25765 129 40 0 25674 0
Current children cumulated CPU time (s) 7.41
Current children cumulated vsize (KiB) 1049212

[startup+3.10069 s]
/proc/loadavg: 27.58 31.06 30.49 6/419 3385
/proc/meminfo: memFree=254429532/264257624 swapFree=67111528/67111528
[pid=3374] ppid=3372 vsize=18324 CPUtime=0 cores=0-31
/proc/3374/stat : 3374 (ppfolio) S 3372 3374 3315 0 -1 4202496 511 0 0 0 0 0 0 0 18 0 1 0 727261750 18763776 385 268435456000 4194304 4287625 140735800438592 18446744073709551615 258650975999 0 0 4096 65536 9170076650967138303 0 0 17 16 0 0 0
/proc/3374/statm: 4581 385 337 23 0 91 0
[pid=3375] ppid=3374 vsize=338640 CPUtime=3.08 cores=0
/proc/3375/stat : 3375 (cryptominisat) R 3374 3374 3315 0 -1 4202496 72647 0 0 0 287 21 0 0 25 0 1 0 727261750 346767360 72006 268435456000 4194304 4587354 140733290898224 18446744073709551615 4238571 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/3375/statm: 84660 72006 303 96 0 81222 0
[pid=3376] ppid=3374 vsize=111764 CPUtime=3.09 cores=1
/proc/3376/stat : 3376 (clasp) R 3374 3374 3315 0 -1 4202496 27442 0 0 0 294 15 0 0 25 0 1 0 727261750 114446336 24961 268435456000 4194304 4785740 140735199264048 18446744073709551615 4553097 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/3376/statm: 27941 24961 334 145 0 24986 0
[pid=3377] ppid=3374 vsize=500316 CPUtime=3.08 cores=2
/proc/3377/stat : 3377 (TNM) R 3374 3374 3315 0 -1 4202496 19646 0 0 0 298 10 0 0 25 0 1 0 727261750 512323584 19545 268435456000 134512640 134534564 4292630352 18446744073709551615 4294960144 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/3377/statm: 125079 19545 90 6 0 123744 0
[pid=3378] ppid=3374 vsize=202116 CPUtime=3.08 cores=3
/proc/3378/stat : 3378 (march_hi) R 3374 3374 3315 0 -1 4194304 48840 0 0 0 286 22 0 0 25 0 1 0 727261750 206966784 48736 268435456000 4194304 4331988 140733561838752 18446744073709551615 258650616632 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/3378/statm: 50529 48736 112 34 0 48971 0
[pid=3379] ppid=3374 vsize=321788 CPUtime=3.09 cores=4-31
/proc/3379/stat : 3379 (plingeling) R 3374 3374 3315 0 -1 4202496 79318 1239 0 0 292 16 0 1 25 0 1 0 727261750 329510912 79106 268435456000 4194304 4356188 140734709006800 18446744073709551615 4274637 0 0 4096 17442 0 0 0 17 10 0 0 0
/proc/3379/statm: 80447 79106 129 40 0 78991 0
Current children cumulated CPU time (s) 15.42
Current children cumulated vsize (KiB) 1492948

[startup+6.30084 s]
/proc/loadavg: 27.58 31.06 30.49 6/419 3385
/proc/meminfo: memFree=253820208/264257624 swapFree=67111528/67111528
[pid=3374] ppid=3372 vsize=18324 CPUtime=0 cores=0-31
/proc/3374/stat : 3374 (ppfolio) S 3372 3374 3315 0 -1 4202496 511 0 0 0 0 0 0 0 18 0 1 0 727261750 18763776 385 268435456000 4194304 4287625 140735800438592 18446744073709551615 258650975999 0 0 4096 65536 9170076650967138303 0 0 17 16 0 0 0
/proc/3374/statm: 4581 385 337 23 0 91 0
[pid=3375] ppid=3374 vsize=338924 CPUtime=6.28 cores=0
/proc/3375/stat : 3375 (cryptominisat) R 3374 3374 3315 0 -1 4202496 72727 0 0 0 607 21 0 0 25 0 1 0 727261750 347058176 72086 268435456000 4194304 4587354 140733290898224 18446744073709551615 4238611 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/3375/statm: 84731 72086 303 96 0 81293 0
[pid=3376] ppid=3374 vsize=119884 CPUtime=6.29 cores=1
/proc/3376/stat : 3376 (clasp) R 3374 3374 3315 0 -1 4202496 29450 0 0 0 613 16 0 0 25 0 1 0 727261750 122761216 26969 268435456000 4194304 4785740 140735199264048 18446744073709551615 4553003 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/3376/statm: 29971 26969 334 145 0 27016 0
[pid=3377] ppid=3374 vsize=714224 CPUtime=6.29 cores=2
/proc/3377/stat : 3377 (TNM) R 3374 3374 3315 0 -1 4202496 82626 0 0 0 603 26 0 0 25 0 1 0 727261750 731365376 82524 268435456000 134512640 134534564 4292630352 18446744073709551615 134524536 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/3377/statm: 178556 82524 91 6 0 177221 0
[pid=3378] ppid=3374 vsize=429372 CPUtime=6.28 cores=3
/proc/3378/stat : 3378 (march_hi) R 3374 3374 3315 0 -1 4194304 78603 0 0 0 599 29 0 0 25 0 1 0 727261750 439676928 60915 268435456000 4194304 4331988 140733561838752 18446744073709551615 4257060 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/3378/statm: 107343 60915 124 34 0 105785 0
[pid=3379] ppid=3374 vsize=608508 CPUtime=6.29 cores=4-31
/proc/3379/stat : 3379 (plingeling) R 3374 3374 3315 0 -1 4202496 150998 1239 0 0 598 30 0 1 25 0 1 0 727261750 623112192 150786 268435456000 4194304 4356188 140734709006800 18446744073709551615 4274637 0 0 4096 17442 0 0 0 17 10 0 0 0
/proc/3379/statm: 152127 150786 129 40 0 150671 0
Current children cumulated CPU time (s) 31.43
Current children cumulated vsize (KiB) 2229236

[startup+12.7011 s]
/proc/loadavg: 24.11 30.20 30.22 6/419 3385
/proc/meminfo: memFree=252915728/264257624 swapFree=67111528/67111528
[pid=3374] ppid=3372 vsize=18324 CPUtime=0 cores=0-31
/proc/3374/stat : 3374 (ppfolio) S 3372 3374 3315 0 -1 4202496 511 0 0 0 0 0 0 0 18 0 1 0 727261750 18763776 385 268435456000 4194304 4287625 140735800438592 18446744073709551615 258650975999 0 0 4096 65536 9170076650967138303 0 0 17 16 0 0 0
/proc/3374/statm: 4581 385 337 23 0 91 0
[pid=3375] ppid=3374 vsize=339348 CPUtime=12.68 cores=0
/proc/3375/stat : 3375 (cryptominisat) R 3374 3374 3315 0 -1 4202496 72859 0 0 0 1247 21 0 0 25 0 1 0 727261750 347492352 72218 268435456000 4194304 4587354 140733290898224 18446744073709551615 4238499 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/3375/statm: 84837 72218 303 96 0 81399 0
[pid=3376] ppid=3374 vsize=130240 CPUtime=12.68 cores=1
/proc/3376/stat : 3376 (clasp) R 3374 3374 3315 0 -1 4202496 31982 0 0 0 1252 16 0 0 25 0 1 0 727261750 133365760 29501 268435456000 4194304 4785740 140735199264048 18446744073709551615 4287028 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/3376/statm: 32560 29501 334 145 0 29605 0
[pid=3377] ppid=3374 vsize=714224 CPUtime=12.69 cores=2
/proc/3377/stat : 3377 (TNM) R 3374 3374 3315 0 -1 4202496 82626 0 0 0 1243 26 0 0 25 0 1 0 727261750 731365376 82524 268435456000 134512640 134534564 4292630352 18446744073709551615 134524752 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/3377/statm: 178556 82524 91 6 0 177221 0
[pid=3378] ppid=3374 vsize=467108 CPUtime=12.69 cores=3
/proc/3378/stat : 3378 (march_hi) R 3374 3374 3315 0 -1 4194304 152413 0 0 0 1224 45 0 0 25 0 1 0 727261750 478318592 74430 268435456000 4194304 4331988 140733561838752 18446744073709551615 258650351330 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/3378/statm: 116777 74430 127 34 0 115219 0
[pid=3379] ppid=3374 vsize=1209752 CPUtime=12.69 cores=4-31
/proc/3379/stat : 3379 (plingeling) R 3374 3374 3315 0 -1 4202496 301309 1239 0 0 1209 59 0 1 25 0 1 0 727261750 1238786048 301097 268435456000 4194304 4356188 140734709006800 18446744073709551615 4277592 0 0 4096 17442 0 0 0 17 10 0 0 0
/proc/3379/statm: 302438 301097 129 40 0 300982 0
Current children cumulated CPU time (s) 63.43
Current children cumulated vsize (KiB) 2878996

[startup+25.5015 s]
/proc/loadavg: 21.17 29.37 29.95 6/419 3385
/proc/meminfo: memFree=252418808/264257624 swapFree=67111528/67111528
[pid=3374] ppid=3372 vsize=18324 CPUtime=0 cores=0-31
/proc/3374/stat : 3374 (ppfolio) S 3372 3374 3315 0 -1 4202496 511 0 0 0 0 0 0 0 15 0 1 0 727261750 18763776 385 268435456000 4194304 4287625 140735800438592 18446744073709551615 258650975999 0 0 4096 65536 9170076650967138303 0 0 17 16 0 0 0
/proc/3374/statm: 4581 385 337 23 0 91 0
[pid=3375] ppid=3374 vsize=554784 CPUtime=25.48 cores=0
/proc/3375/stat : 3375 (cryptominisat) R 3374 3374 3315 0 -1 4202496 118199 0 0 0 2517 31 0 0 25 0 1 0 727261750 568098816 116555 268435456000 4194304 4587354 140733290898224 18446744073709551615 258650653319 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/3375/statm: 138696 116555 312 96 0 135258 0
[pid=3376] ppid=3374 vsize=158956 CPUtime=25.49 cores=1
/proc/3376/stat : 3376 (clasp) R 3374 3374 3315 0 -1 4202496 39082 0 0 0 2530 19 0 0 25 0 1 0 727261750 162770944 36601 268435456000 4194304 4785740 140735199264048 18446744073709551615 4553402 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/3376/statm: 39739 36601 334 145 0 36784 0
[pid=3377] ppid=3374 vsize=714224 CPUtime=25.48 cores=2
/proc/3377/stat : 3377 (TNM) R 3374 3374 3315 0 -1 4202496 82626 0 0 0 2522 26 0 0 25 0 1 0 727261750 731365376 82524 268435456000 134512640 134534564 4292630352 18446744073709551615 134524568 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/3377/statm: 178556 82524 91 6 0 177221 0
[pid=3378] ppid=3374 vsize=415932 CPUtime=25.48 cores=3
/proc/3378/stat : 3378 (march_hi) R 3374 3374 3315 0 -1 4202496 347503 0 0 0 2458 90 0 0 25 0 1 0 727261750 425914368 61668 268435456000 4194304 4331988 140733561838752 18446744073709551615 4256854 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/3378/statm: 103983 61668 148 34 0 102425 0
[pid=3379] ppid=3374 vsize=1494340 CPUtime=25.48 cores=4-31
/proc/3379/stat : 3379 (plingeling) R 3374 3374 3315 0 -1 4202496 372509 1239 0 0 2471 76 0 1 25 0 1 0 727261750 1530204160 372244 268435456000 4194304 4356188 140734709006800 18446744073709551615 258650955760 0 0 4096 17442 0 0 0 17 10 0 0 0
/proc/3379/statm: 373585 372244 129 40 0 372129 0
Current children cumulated CPU time (s) 127.41
Current children cumulated vsize (KiB) 3356560

[startup+51.1023 s]
/proc/loadavg: 17.41 27.77 29.40 33/447 3417
/proc/meminfo: memFree=249535408/264257624 swapFree=67111528/67111528
[pid=3374] ppid=3372 vsize=18324 CPUtime=0 cores=0-31
/proc/3374/stat : 3374 (ppfolio) S 3372 3374 3315 0 -1 4202496 511 0 0 0 0 0 0 0 15 0 1 0 727261750 18763776 385 268435456000 4194304 4287625 140735800438592 18446744073709551615 258650975999 0 0 4096 65536 9170076650967138303 0 0 17 16 0 0 0
/proc/3374/statm: 4581 385 337 23 0 91 0
[pid=3375] ppid=3374 vsize=764096 CPUtime=51.08 cores=0
/proc/3375/stat : 3375 (cryptominisat) R 3374 3374 3315 0 -1 4202496 264439 0 0 0 5043 65 0 0 25 0 1 0 727261750 782434304 169095 268435456000 4194304 4587354 140733290898224 18446744073709551615 4241155 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/3375/statm: 191024 169095 347 96 0 187586 0
[pid=3376] ppid=3374 vsize=189212 CPUtime=51.07 cores=1
/proc/3376/stat : 3376 (clasp) R 3374 3374 3315 0 -1 4202496 46580 0 0 0 5085 22 0 0 25 0 1 0 727261750 193753088 44099 268435456000 4194304 4785740 140735199264048 18446744073709551615 4552628 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/3376/statm: 47303 44099 334 145 0 44348 0
[pid=3377] ppid=3374 vsize=714224 CPUtime=51.08 cores=2
/proc/3377/stat : 3377 (TNM) R 3374 3374 3315 0 -1 4202496 82626 0 0 0 5082 26 0 0 25 0 1 0 727261750 731365376 82524 268435456000 134512640 134534564 4292630352 18446744073709551615 134524686 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/3377/statm: 178556 82524 91 6 0 177221 0
[pid=3378] ppid=3374 vsize=480720 CPUtime=51.08 cores=3

################
# More data... #
################

/proc/3375/statm: 611854 172054 336 96 0 608416 0
[pid=3376] ppid=3374 vsize=6287452 CPUtime=5043.16 cores=1
/proc/3376/stat : 3376 (clasp) R 3374 3374 3315 0 -1 4202496 1555293 0 0 0 503665 651 0 0 25 0 1 0 727261750 6438350848 1548481 268435456000 4194304 4785740 140735199264048 18446744073709551615 4552646 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/3376/statm: 1571863 1548481 334 145 0 1568908 0
[pid=3377] ppid=3374 vsize=793120 CPUtime=5042.78 cores=2
/proc/3377/stat : 3377 (TNM) R 3374 3374 3315 0 -1 4202496 125437 0 0 0 504244 34 0 0 25 0 1 0 727261750 812154880 125335 268435456000 134512640 134534564 4292630352 18446744073709551615 134528302 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/3377/statm: 198280 125335 95 6 0 196945 0
[pid=3378] ppid=3374 vsize=8760404 CPUtime=5043.04 cores=3
/proc/3378/stat : 3378 (march_hi) R 3374 3374 3315 0 -1 4202496 2468643 0 0 0 503020 1284 0 0 25 0 1 0 727261750 8970653696 2130868 268435456000 4194304 4331988 140733561838752 18446744073709551615 4226645 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/3378/statm: 2190101 2130868 149 34 0 2188543 0
[pid=3379] ppid=3374 vsize=6934168 CPUtime=139830 cores=4-31
/proc/3379/stat : 3379 (plingeling) S 3374 3374 3315 0 -1 4202496 17482375 1239 0 0 13970024 13004 0 1 21 0 29 0 727261750 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 12 0 0 0
/proc/3379/statm: 1733542 1601736 159 40 0 1732086 0
[pid=3379/tid=3388] ppid=3374 vsize=6934168 CPUtime=4989.63 cores=4-31
/proc/3379/task/3388/stat : 3388 (plingeling) R 3374 3374 3315 0 -1 4202560 546667 1239 0 0 498593 369 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331002 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=3379/tid=3389] ppid=3374 vsize=6934168 CPUtime=4989.61 cores=4-31
/proc/3379/task/3389/stat : 3389 (plingeling) R 3374 3374 3315 0 -1 4202560 560967 1239 0 0 498564 396 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=3379/tid=3390] ppid=3374 vsize=6934168 CPUtime=4997.19 cores=4-31
/proc/3379/task/3390/stat : 3390 (plingeling) R 3374 3374 3315 0 -1 4202560 566154 1239 0 0 499167 551 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331039 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=3379/tid=3391] ppid=3374 vsize=6934168 CPUtime=4997.77 cores=4-31
/proc/3379/task/3391/stat : 3391 (plingeling) R 3374 3374 3315 0 -1 4202560 694699 1239 0 0 499138 638 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4241640 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=3379/tid=3392] ppid=3374 vsize=6934168 CPUtime=4998.02 cores=4-31
/proc/3379/task/3392/stat : 3392 (plingeling) R 3374 3374 3315 0 -1 4202560 558006 1239 0 0 499380 421 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=3379/tid=3393] ppid=3374 vsize=6934168 CPUtime=4992.94 cores=4-31
/proc/3379/task/3393/stat : 3393 (plingeling) R 3374 3374 3315 0 -1 4202560 595379 1239 0 0 498765 528 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4210856 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=3379/tid=3394] ppid=3374 vsize=6934168 CPUtime=4990.97 cores=4-31
/proc/3379/task/3394/stat : 3394 (plingeling) R 3374 3374 3315 0 -1 4202560 520706 1239 0 0 498730 366 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331019 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3379/tid=3395] ppid=3374 vsize=6934168 CPUtime=4997.27 cores=4-31
/proc/3379/task/3395/stat : 3395 (plingeling) R 3374 3374 3315 0 -1 4202560 517727 1239 0 0 499441 285 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=3379/tid=3396] ppid=3374 vsize=6934168 CPUtime=4983.2 cores=4-31
/proc/3379/task/3396/stat : 3396 (plingeling) R 3374 3374 3315 0 -1 4202560 544632 1239 0 0 497963 356 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4330862 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=3379/tid=3397] ppid=3374 vsize=6934168 CPUtime=4997.63 cores=4-31
/proc/3379/task/3397/stat : 3397 (plingeling) R 3374 3374 3315 0 -1 4202560 529085 1239 0 0 499439 323 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331360 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=3379/tid=3398] ppid=3374 vsize=6934168 CPUtime=4998.27 cores=4-31
/proc/3379/task/3398/stat : 3398 (plingeling) R 3374 3374 3315 0 -1 4202560 644774 1239 0 0 499260 566 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4210721 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=3379/tid=3399] ppid=3374 vsize=6934168 CPUtime=4995.47 cores=4-31
/proc/3379/task/3399/stat : 3399 (plingeling) R 3374 3374 3315 0 -1 4202560 691994 1239 0 0 498816 730 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331015 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=3379/tid=3400] ppid=3374 vsize=6934168 CPUtime=4997.95 cores=4-31
/proc/3379/task/3400/stat : 3400 (plingeling) R 3374 3374 3315 0 -1 4202560 550029 1239 0 0 499399 395 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=3379/tid=3401] ppid=3374 vsize=6934168 CPUtime=4995.4 cores=4-31
/proc/3379/task/3401/stat : 3401 (plingeling) R 3374 3374 3315 0 -1 4202560 603328 1239 0 0 498959 580 0 1 25 0 29 0 727265961 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4330862 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=3379/tid=3402] ppid=3374 vsize=6934168 CPUtime=4997.72 cores=4-31
/proc/3379/task/3402/stat : 3402 (plingeling) R 3374 3374 3315 0 -1 4202560 570654 1239 0 0 499392 379 0 1 25 0 29 0 727265961 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331076 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=3379/tid=3403] ppid=3374 vsize=6934168 CPUtime=4982.26 cores=4-31
/proc/3379/task/3403/stat : 3403 (plingeling) R 3374 3374 3315 0 -1 4202560 554308 1239 0 0 497784 441 0 1 25 0 29 0 727265961 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4210834 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=3379/tid=3404] ppid=3374 vsize=6934168 CPUtime=4997.78 cores=4-31
/proc/3379/task/3404/stat : 3404 (plingeling) R 3374 3374 3315 0 -1 4202560 616503 1239 0 0 499308 469 0 1 25 0 29 0 727265961 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4250495 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=3379/tid=3407] ppid=3374 vsize=6934168 CPUtime=4997.55 cores=4-31
/proc/3379/task/3407/stat : 3407 (plingeling) R 3374 3374 3315 0 -1 4202560 644751 1239 0 0 499327 427 0 1 25 0 29 0 727265961 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=3379/tid=3408] ppid=3374 vsize=6934168 CPUtime=4992.11 cores=4-31
/proc/3379/task/3408/stat : 3408 (plingeling) R 3374 3374 3315 0 -1 4202560 609593 1239 0 0 498652 558 0 1 25 0 29 0 727265961 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=3379/tid=3409] ppid=3374 vsize=6934168 CPUtime=4997.06 cores=4-31
/proc/3379/task/3409/stat : 3409 (plingeling) R 3374 3374 3315 0 -1 4202560 726557 1239 0 0 499017 688 0 1 25 0 29 0 727265962 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=3379/tid=3410] ppid=3374 vsize=6934168 CPUtime=4985.58 cores=4-31
/proc/3379/task/3410/stat : 3410 (plingeling) R 3374 3374 3315 0 -1 4202560 690903 1239 0 0 498154 403 0 1 25 0 29 0 727265962 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331311 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=3379/tid=3411] ppid=3374 vsize=6934168 CPUtime=4973.88 cores=4-31
/proc/3379/task/3411/stat : 3411 (plingeling) R 3374 3374 3315 0 -1 4202560 578342 1239 0 0 496925 462 0 1 25 0 29 0 727265963 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4334437 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=3379/tid=3412] ppid=3374 vsize=6934168 CPUtime=4997.4 cores=4-31
/proc/3379/task/3412/stat : 3412 (plingeling) R 3374 3374 3315 0 -1 4202560 544452 1239 0 0 499368 371 0 1 25 0 29 0 727265963 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331002 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=3379/tid=3413] ppid=3374 vsize=6934168 CPUtime=4997.58 cores=4-31
/proc/3379/task/3413/stat : 3413 (plingeling) R 3374 3374 3315 0 -1 4202560 678408 1239 0 0 499374 383 0 1 25 0 29 0 727265963 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4258634 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=3379/tid=3414] ppid=3374 vsize=6934168 CPUtime=4989.92 cores=4-31
/proc/3379/task/3414/stat : 3414 (plingeling) R 3374 3374 3315 0 -1 4202560 671870 1239 0 0 498607 384 0 1 25 0 29 0 727265963 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331047 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=3379/tid=3415] ppid=3374 vsize=6934168 CPUtime=4998.48 cores=4-31
/proc/3379/task/3415/stat : 3415 (plingeling) R 3374 3374 3315 0 -1 4202560 540707 1239 0 0 499573 274 0 1 25 0 29 0 727265963 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4279677 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=3379/tid=3416] ppid=3374 vsize=6934168 CPUtime=4967.34 cores=4-31
/proc/3379/task/3416/stat : 3416 (plingeling) R 3374 3374 3315 0 -1 4202560 601299 1239 0 0 496274 459 0 1 25 0 29 0 727265963 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4210834 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=3379/tid=3417] ppid=3374 vsize=6934168 CPUtime=4992.42 cores=4-31
/proc/3379/task/3417/stat : 3417 (plingeling) R 3374 3374 3315 0 -1 4202560 700357 1239 0 0 498579 662 0 1 25 0 29 0 727265963 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331015 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 160002
Current children cumulated vsize (KiB) 25240884

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5044.1 s]
/proc/loadavg: 32.04 32.06 32.00 33/447 3562
/proc/meminfo: memFree=232803204/264257624 swapFree=67111528/67111528
[pid=3374] ppid=3372 vsize=18324 CPUtime=0 cores=0-31
/proc/3374/stat : 3374 (ppfolio) S 3372 3374 3315 0 -1 4202496 511 0 0 0 0 0 0 0 15 0 1 0 727261750 18763776 385 268435456000 4194304 4287625 140735800438592 18446744073709551615 258650975999 0 0 4096 65536 9170076650967138303 0 0 17 16 0 0 0
/proc/3374/statm: 4581 385 337 23 0 91 0
[pid=3375] ppid=3374 vsize=2447416 CPUtime=5043.15 cores=0
/proc/3375/stat : 3375 (cryptominisat) R 3374 3374 3315 0 -1 4202496 2553735 0 0 0 503567 748 0 0 25 0 1 0 727261750 2506153984 172054 268435456000 4194304 4587354 140733290898224 18446744073709551615 4238271 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/3375/statm: 611854 172054 336 96 0 608416 0
[pid=3376] ppid=3374 vsize=6287452 CPUtime=5043.16 cores=1
/proc/3376/stat : 3376 (clasp) R 3374 3374 3315 0 -1 4202496 1555293 0 0 0 503665 651 0 0 25 0 1 0 727261750 6438350848 1548481 268435456000 4194304 4785740 140735199264048 18446744073709551615 4552646 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/3376/statm: 1571863 1548481 334 145 0 1568908 0
[pid=3377] ppid=3374 vsize=793120 CPUtime=5042.78 cores=2
/proc/3377/stat : 3377 (TNM) R 3374 3374 3315 0 -1 4202496 125437 0 0 0 504244 34 0 0 25 0 1 0 727261750 812154880 125335 268435456000 134512640 134534564 4292630352 18446744073709551615 134528302 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/3377/statm: 198280 125335 95 6 0 196945 0
[pid=3378] ppid=3374 vsize=8760404 CPUtime=5043.04 cores=3
/proc/3378/stat : 3378 (march_hi) R 3374 3374 3315 0 -1 4202496 2468643 0 0 0 503020 1284 0 0 25 0 1 0 727261750 8970653696 2130868 268435456000 4194304 4331988 140733561838752 18446744073709551615 4226645 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/3378/statm: 2190101 2130868 149 34 0 2188543 0
[pid=3379] ppid=3374 vsize=6934168 CPUtime=139830 cores=4-31
/proc/3379/stat : 3379 (plingeling) S 3374 3374 3315 0 -1 4202496 17482375 1239 0 0 13970024 13004 0 1 21 0 29 0 727261750 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 12 0 0 0
/proc/3379/statm: 1733542 1601736 159 40 0 1732086 0
[pid=3379/tid=3388] ppid=3374 vsize=6934168 CPUtime=4989.63 cores=4-31
/proc/3379/task/3388/stat : 3388 (plingeling) R 3374 3374 3315 0 -1 4202560 546667 1239 0 0 498593 369 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331002 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=3379/tid=3389] ppid=3374 vsize=6934168 CPUtime=4989.61 cores=4-31
/proc/3379/task/3389/stat : 3389 (plingeling) R 3374 3374 3315 0 -1 4202560 560967 1239 0 0 498564 396 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=3379/tid=3390] ppid=3374 vsize=6934168 CPUtime=4997.19 cores=4-31
/proc/3379/task/3390/stat : 3390 (plingeling) R 3374 3374 3315 0 -1 4202560 566154 1239 0 0 499167 551 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331039 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=3379/tid=3391] ppid=3374 vsize=6934168 CPUtime=4997.77 cores=4-31
/proc/3379/task/3391/stat : 3391 (plingeling) R 3374 3374 3315 0 -1 4202560 694699 1239 0 0 499138 638 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4241640 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=3379/tid=3392] ppid=3374 vsize=6934168 CPUtime=4998.02 cores=4-31
/proc/3379/task/3392/stat : 3392 (plingeling) R 3374 3374 3315 0 -1 4202560 558006 1239 0 0 499380 421 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=3379/tid=3393] ppid=3374 vsize=6934168 CPUtime=4992.94 cores=4-31
/proc/3379/task/3393/stat : 3393 (plingeling) R 3374 3374 3315 0 -1 4202560 595379 1239 0 0 498765 528 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4210856 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=3379/tid=3394] ppid=3374 vsize=6934168 CPUtime=4990.97 cores=4-31
/proc/3379/task/3394/stat : 3394 (plingeling) R 3374 3374 3315 0 -1 4202560 520706 1239 0 0 498730 366 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331019 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3379/tid=3395] ppid=3374 vsize=6934168 CPUtime=4997.27 cores=4-31
/proc/3379/task/3395/stat : 3395 (plingeling) R 3374 3374 3315 0 -1 4202560 517727 1239 0 0 499441 285 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=3379/tid=3396] ppid=3374 vsize=6934168 CPUtime=4983.2 cores=4-31
/proc/3379/task/3396/stat : 3396 (plingeling) R 3374 3374 3315 0 -1 4202560 544632 1239 0 0 497963 356 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4330862 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=3379/tid=3397] ppid=3374 vsize=6934168 CPUtime=4997.63 cores=4-31
/proc/3379/task/3397/stat : 3397 (plingeling) R 3374 3374 3315 0 -1 4202560 529085 1239 0 0 499439 323 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331360 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=3379/tid=3398] ppid=3374 vsize=6934168 CPUtime=4998.27 cores=4-31
/proc/3379/task/3398/stat : 3398 (plingeling) R 3374 3374 3315 0 -1 4202560 644774 1239 0 0 499260 566 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4210721 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=3379/tid=3399] ppid=3374 vsize=6934168 CPUtime=4995.47 cores=4-31
/proc/3379/task/3399/stat : 3399 (plingeling) R 3374 3374 3315 0 -1 4202560 691994 1239 0 0 498816 730 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331015 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=3379/tid=3400] ppid=3374 vsize=6934168 CPUtime=4997.95 cores=4-31
/proc/3379/task/3400/stat : 3400 (plingeling) R 3374 3374 3315 0 -1 4202560 550029 1239 0 0 499399 395 0 1 25 0 29 0 727265960 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=3379/tid=3401] ppid=3374 vsize=6934168 CPUtime=4995.4 cores=4-31
/proc/3379/task/3401/stat : 3401 (plingeling) R 3374 3374 3315 0 -1 4202560 603328 1239 0 0 498959 580 0 1 25 0 29 0 727265961 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4330862 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=3379/tid=3402] ppid=3374 vsize=6934168 CPUtime=4997.72 cores=4-31
/proc/3379/task/3402/stat : 3402 (plingeling) R 3374 3374 3315 0 -1 4202560 570654 1239 0 0 499392 379 0 1 25 0 29 0 727265961 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331076 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=3379/tid=3403] ppid=3374 vsize=6934168 CPUtime=4982.26 cores=4-31
/proc/3379/task/3403/stat : 3403 (plingeling) R 3374 3374 3315 0 -1 4202560 554308 1239 0 0 497784 441 0 1 25 0 29 0 727265961 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4210834 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=3379/tid=3404] ppid=3374 vsize=6934168 CPUtime=4997.78 cores=4-31
/proc/3379/task/3404/stat : 3404 (plingeling) R 3374 3374 3315 0 -1 4202560 616503 1239 0 0 499308 469 0 1 25 0 29 0 727265961 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4250495 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=3379/tid=3407] ppid=3374 vsize=6934168 CPUtime=4997.55 cores=4-31
/proc/3379/task/3407/stat : 3407 (plingeling) R 3374 3374 3315 0 -1 4202560 644751 1239 0 0 499327 427 0 1 25 0 29 0 727265961 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=3379/tid=3408] ppid=3374 vsize=6934168 CPUtime=4992.11 cores=4-31
/proc/3379/task/3408/stat : 3408 (plingeling) R 3374 3374 3315 0 -1 4202560 609593 1239 0 0 498652 558 0 1 25 0 29 0 727265961 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=3379/tid=3409] ppid=3374 vsize=6934168 CPUtime=4997.06 cores=4-31
/proc/3379/task/3409/stat : 3409 (plingeling) R 3374 3374 3315 0 -1 4202560 726557 1239 0 0 499017 688 0 1 25 0 29 0 727265962 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=3379/tid=3410] ppid=3374 vsize=6934168 CPUtime=4985.58 cores=4-31
/proc/3379/task/3410/stat : 3410 (plingeling) R 3374 3374 3315 0 -1 4202560 690903 1239 0 0 498154 403 0 1 25 0 29 0 727265962 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331311 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=3379/tid=3411] ppid=3374 vsize=6934168 CPUtime=4973.88 cores=4-31
/proc/3379/task/3411/stat : 3411 (plingeling) R 3374 3374 3315 0 -1 4202560 578342 1239 0 0 496925 462 0 1 25 0 29 0 727265963 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4334437 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=3379/tid=3412] ppid=3374 vsize=6934168 CPUtime=4997.4 cores=4-31
/proc/3379/task/3412/stat : 3412 (plingeling) R 3374 3374 3315 0 -1 4202560 544452 1239 0 0 499368 371 0 1 25 0 29 0 727265963 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331002 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=3379/tid=3413] ppid=3374 vsize=6934168 CPUtime=4997.58 cores=4-31
/proc/3379/task/3413/stat : 3413 (plingeling) R 3374 3374 3315 0 -1 4202560 678408 1239 0 0 499374 383 0 1 25 0 29 0 727265963 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4258634 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=3379/tid=3414] ppid=3374 vsize=6934168 CPUtime=4989.92 cores=4-31
/proc/3379/task/3414/stat : 3414 (plingeling) R 3374 3374 3315 0 -1 4202560 671870 1239 0 0 498607 384 0 1 25 0 29 0 727265963 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331047 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=3379/tid=3415] ppid=3374 vsize=6934168 CPUtime=4998.48 cores=4-31
/proc/3379/task/3415/stat : 3415 (plingeling) R 3374 3374 3315 0 -1 4202560 540707 1239 0 0 499573 274 0 1 25 0 29 0 727265963 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4279677 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=3379/tid=3416] ppid=3374 vsize=6934168 CPUtime=4967.34 cores=4-31
/proc/3379/task/3416/stat : 3416 (plingeling) R 3374 3374 3315 0 -1 4202560 601299 1239 0 0 496274 459 0 1 25 0 29 0 727265963 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4210834 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=3379/tid=3417] ppid=3374 vsize=6934168 CPUtime=4992.42 cores=4-31
/proc/3379/task/3417/stat : 3417 (plingeling) R 3374 3374 3315 0 -1 4202560 700357 1239 0 0 498579 662 0 1 25 0 29 0 727265963 7100588032 1601736 268435456000 4194304 4356188 140734709006800 18446744073709551615 4331015 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 160002
Current children cumulated vsize (KiB) 25240884

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 3374 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=2999
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=3999
# CPU time returned by wait4() is 0.006998
# while last known CPU time is 160002
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5044.14
CPU time (s): 160002
CPU user time (s): 159845
CPU system time (s): 157.22
CPU usage (%): 3172.05
Max. virtual memory (cumulated for all children) (KiB): 27403684

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.002999
system time used= 0.003999
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 511
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 309
involuntary context switches= 2

runsolver used 27.7308 second user time and 114.341 second system time

The end

Launcher Data

Begin job on node200 at 2011-05-23 15:35:53
IDJOB=3347731
IDBENCH=88163
IDSOLVER=1808
FILE ID=node200/3347731-1306157750
RUNJOBID= node200-1306157750-3329
PBS_JOBID= 13387850
Free space on /tmp= 61968 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT-Race-2010-CNF/hardware-verification/velev/11pipe_11_ooo.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3347731-1306157750/watcher-3347731-1306157750 -o /tmp/evaluation-result-3347731-1306157750/solver-3347731-1306157750 -C 160000 -W 5100 -M 266338304  ppfolio HOME/instance-3347731-1306157750.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 266338304 MiB
NBCORE= 32

MD5SUM BENCH= f603f11fbfe855df381282b7bce010ea
RANDOM SEED=387176305

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.98
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.23
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      255152032 kB
Buffers:        461472 kB
Cached:        8228516 kB
SwapCached:          0 kB
Active:        7770240 kB
Inactive:       988932 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      255152032 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           91220 kB
Writeback:           0 kB
AnonPages:       69016 kB
Mapped:          14480 kB
Slab:           255072 kB
PageTables:       4432 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   191892 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 61956 MiB
End job on node200 at 2011-05-23 17:00:00