Trace number 3385371

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? (MO) 9490.66 1194.72

General information on the benchmark

Namerandom/large/
unif-k5-r20-v2000-c40000-S971556279-001-UNKNOWN.cnf
MD5SUM9224277e61359574518ca553717e02ee
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark640.831
Satisfiable
(Un)Satisfiability was proved
Number of variables2000
Number of clauses40000
Sum of the clauses size200000
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 540000
Number of clauses of size over 50

Solver Data 

0.30/0.47	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.30/0.47	c INFO: solving HOME/instance-3385371-1305565937.cnf
0.30/0.47	c INFO: computing task features
0.49/0.68	c INFO: computed variable-clause graph statistics
0.49/0.69	c INFO: computed clause balance statistics
1.59/1.71	c INFO: computed variable balance statistics
2.10/2.29	c INFO: computed small-clause counts
2.10/2.29	c INFO: computed Horn-clause counts
2.19/2.34	c INFO: computed variable graph statistics
2.19/2.34	c INFO: collected features for HOME/instance-3385371-1305565937.cnf in 1.87 s
2.19/2.37	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.38)
2.28/2.40	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.13)
2.28/2.43	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.08)
2.28/2.46	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.07)
2.28/2.49	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.05)
2.38/2.52	c INFO: running gnovelty+2@0 for 97 with 4997 remaining (b = 0.05)
2.38/2.55	c INFO: running FH@0 for 97 with 4997 remaining (b = 0.03)
2.38/2.59	c INFO: running adaptg2wsat2009++@0 for 97 with 4997 remaining (b = 0.03)
796.70/102.05	c INFO: running adaptg2wsat2009++@0 for 97 with 4898 remaining (b = 0.02)
803.07/102.84	c INFO: running FH@0 for 97 with 4897 remaining (b = 0.02)
803.07/102.88	c INFO: running hybridGM3@0 for 194 with 4897 remaining (b = 0.04)
803.75/102.92	c INFO: running gnovelty+2@0 for 97 with 4897 remaining (b = 0.02)
803.75/102.97	c INFO: running hybridGM3@0 for 194 with 4897 remaining (b = 0.03)
804.11/103.01	c INFO: running adaptg2wsat2009++@0 for 97 with 4897 remaining (b = 0.01)
804.11/103.06	c INFO: running hybridGM3@0 for 194 with 4897 remaining (b = 0.03)
804.31/103.11	c INFO: running hybridGM3@0 for 194 with 4897 remaining (b = 0.02)
1596.40/202.47	c INFO: running TNM@0 for 1552 with 4798 remaining (b = 0.19)
1601.93/203.30	c INFO: running FH@0 for 97 with 4797 remaining (b = 0.01)
1602.65/203.41	c INFO: running adaptg2wsat2009++@0 for 97 with 4797 remaining (b = 0.01)
1604.33/203.46	c INFO: running adaptg2wsat2009++@0 for 97 with 4797 remaining (b = 0.01)
2356.83/297.98	c INFO: running hybridGM3@100 for 97 with 4702 remaining (b = 0.01)
2371.95/299.84	c INFO: running hybridGM3@100 for 97 with 4700 remaining (b = 0.01)
2372.75/299.97	c INFO: running hybridGM3@100 for 97 with 4700 remaining (b = 0.01)
2379.90/300.84	c INFO: running adaptg2wsat2009++@0 for 97 with 4699 remaining (b = 0.01)
2402.99/303.73	c INFO: running adaptg2wsat2009++@97 for 97 with 4696 remaining (b = 0.01)
2411.08/304.79	c INFO: running hybridGM3@99 for 97 with 4695 remaining (b = 0.01)
2412.52/304.92	c INFO: running hybridGM3@0 for 194 with 4695 remaining (b = 0.02)
3131.66/395.24	c INFO: running hybridGM3@296 for 97 with 4605 remaining (b = 0.01)
3132.45/395.37	c INFO: running TNM@0 for 1455 with 4605 remaining (b = 0.14)
3147.56/397.27	c INFO: running adaptg2wsat2009++@100 for 97 with 4603 remaining (b = 0.01)
3165.06/399.43	c INFO: running hybridGM3@297 for 97 with 4601 remaining (b = 0.01)
3178.57/401.16	c INFO: running iPAWS@0 for 97 with 4599 remaining (b = 0.01)
3219.17/406.22	c INFO: running FH@0 for 291 with 4594 remaining (b = 0.03)
3915.97/493.78	c INFO: running gnovelty+2@0 for 97 with 4506 remaining (b = 0.01)
3916.85/493.88	c INFO: running hybridGM3@394 for 97 with 4506 remaining (b = 0.01)
3939.05/496.68	c INFO: running hybridGM3@391 for 97 with 4503 remaining (b = 0.01)
3970.84/500.63	c INFO: running adaptg2wsat2009++@100 for 97 with 4499 remaining (b = 0.01)
4004.44/504.88	c INFO: running hybridGM3@94 for 97 with 4495 remaining (b = 0.01)
4698.66/592.09	c INFO: running hybridGM3@0 for 194 with 4408 remaining (b = 0.02)
4716.91/594.32	c INFO: running hybridGM3@0 for 194 with 4406 remaining (b = 0.02)
4724.83/595.32	c INFO: running hybridGM3@0 for 194 with 4405 remaining (b = 0.01)
4746.31/598.07	c INFO: running TNM@0 for 1552 with 4402 remaining (b = 0.11)
4812.23/606.31	c INFO: running hybridGM3@189 for 97 with 4394 remaining (b = 0.01)
5571.22/701.73	c INFO: running adaptg2wsat2009++@294 for 291 with 4298 remaining (b = 0.02)
5575.86/702.40	c INFO: running FH@100 for 291 with 4298 remaining (b = 0.02)
6257.56/788.09	c INFO: running iPAWS@0 for 97 with 4212 remaining (b = 0.01)
6267.08/789.29	c INFO: running hybridGM3@100 for 97 with 4211 remaining (b = 0.01)
6282.98/791.29	c INFO: running hybridGM3@101 for 97 with 4209 remaining (b = 0.01)
7056.87/888.55	c INFO: running hybridGM3@96 for 97 with 4111 remaining (b = 0.01)
7058.58/888.75	c INFO: running hybridGM3@296 for 194 with 4111 remaining (b = 0.02)
7074.37/890.75	c INFO: running adaptg2wsat2009++@294 for 291 with 4109 remaining (b = 0.02)
7831.55/986.00	c INFO: running FH@0 for 97 with 4014 remaining (b = 0.01)
7871.30/990.94	c INFO: running NCVWr@0 for 485 with 4009 remaining (b = 0.03)
7923.78/997.61	c INFO: running FH@295 for 97 with 4002 remaining (b = 0.01)
8631.63/1086.56	c INFO: running hybridGM3@395 for 97 with 3913 remaining (b = 0.01)
8678.51/1092.48	c INFO: running FH@289 for 194 with 3908 remaining (b = 0.01)
8683.28/1093.03	c INFO: running gnovelty+2@0 for 97 with 3907 remaining (b = 0.01)
9382.30/1180.91	c INFO: running adaptg2wsat2009++@391 for 194 with 3819 remaining (b = 0.01)
9430.79/1187.02	c INFO: running hybridGM3@0 for 194 with 3813 remaining (b = 0.01)
9482.56/1193.55	c INFO: running iPAWS@0 for 97 with 3806 remaining (b = 0.01)
9490.66/1194.67	c INFO: running hybridGM3@0 for 194 with 3805 remaining (b = 0.01)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385371-1305565937/watcher-3385371-1305565937 -o /tmp/evaluation-result-3385371-1305565937/solver-3385371-1305565937 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385371-1305565937.cnf -seed 1655447407 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.10 8.08 5.38 1/464 22121
/proc/meminfo: memFree=24644548/32951124 swapFree=60609076/67111528
[pid=22121] ppid=22119 vsize=16236 CPUtime=0 cores=0-7
/proc/22121/stat : 22121 (python) R 22119 22121 21743 0 -1 4202496 1049 0 0 0 0 0 0 0 17 0 1 0 668194715 16625664 921 33554432000 4194304 5683644 140735217443040 18446744073709551615 5293775 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/22121/statm: 4059 921 381 364 0 546 0

[startup+0.086848 s]
/proc/loadavg: 8.10 8.08 5.38 1/464 22121
/proc/meminfo: memFree=24644548/32951124 swapFree=60609076/67111528
[pid=22121] ppid=22119 vsize=93960 CPUtime=0.05 cores=0-7
/proc/22121/stat : 22121 (python) R 22119 22121 21743 0 -1 4202496 2695 0 0 0 4 1 0 0 17 0 1 0 668194715 96215040 2407 33554432000 4194304 5683644 140735217443040 18446744073709551615 4620076 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/22121/statm: 23490 2407 764 364 0 1622 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 93960

[startup+0.100849 s]
/proc/loadavg: 8.10 8.08 5.38 1/464 22121
/proc/meminfo: memFree=24644548/32951124 swapFree=60609076/67111528
[pid=22121] ppid=22119 vsize=104216 CPUtime=0.07 cores=0-7
/proc/22121/stat : 22121 (python) R 22119 22121 21743 0 -1 4202496 3046 0 0 0 5 2 0 0 17 0 1 0 668194715 106717184 2731 33554432000 4194304 5683644 140735217443040 18446744073709551615 270613108736 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/22121/statm: 26054 2731 847 364 0 1802 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 104216

[startup+0.300799 s]
/proc/loadavg: 8.10 8.08 5.38 1/464 22121
/proc/meminfo: memFree=24644548/32951124 swapFree=60609076/67111528
[pid=22121] ppid=22119 vsize=168668 CPUtime=0.23 cores=0-7
/proc/22121/stat : 22121 (python) R 22119 22121 21743 0 -1 4202496 6721 932 1 0 17 6 0 0 17 0 1 0 668194715 172716032 4796 33554432000 4194304 5683644 140735217443040 18446744073709551615 270613110416 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/22121/statm: 42167 4796 1331 364 0 3452 0
Current children cumulated CPU time (s) 0.23
Current children cumulated vsize (KiB) 168668

[startup+0.700694 s]
/proc/loadavg: 8.10 8.08 5.38 1/464 22121
/proc/meminfo: memFree=24644548/32951124 swapFree=60609076/67111528
[pid=22121] ppid=22119 vsize=290804 CPUtime=0.59 cores=0-7
/proc/22121/stat : 22121 (python) R 22119 22121 21743 0 -1 4202496 14008 932 1 0 51 8 0 0 17 0 1 0 668194715 297783296 10085 33554432000 4194304 5683644 140735217443040 18446744073709551615 270616519337 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/22121/statm: 72701 10085 1826 364 0 8625 0
Current children cumulated CPU time (s) 0.59
Current children cumulated vsize (KiB) 290804

[startup+1.50056 s]
/proc/loadavg: 8.10 8.08 5.38 2/465 22126
/proc/meminfo: memFree=24611060/32951124 swapFree=60609076/67111528
[pid=22121] ppid=22119 vsize=290804 CPUtime=1.39 cores=0-7
/proc/22121/stat : 22121 (python) R 22119 22121 21743 0 -1 4202496 14008 932 1 0 131 8 0 0 19 0 1 0 668194715 297783296 10085 33554432000 4194304 5683644 140735217443040 18446744073709551615 47032278463776 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/22121/statm: 72701 10085 1826 364 0 8625 0
Current children cumulated CPU time (s) 1.39
Current children cumulated vsize (KiB) 290804

[startup+3.10023 s]
/proc/loadavg: 8.10 8.08 5.38 2/465 22126
/proc/meminfo: memFree=24610564/32951124 swapFree=60609076/67111528
[pid=22121] ppid=22119 vsize=432072 CPUtime=2.48 cores=0-7
/proc/22121/stat : 22121 (python) S 22119 22121 21743 0 -1 4202496 19880 932 1 0 238 10 0 0 19 0 9 0 668194715 442441728 8766 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624937419 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/22121/statm: 108018 8766 1891 364 0 43915 0
Current children cumulated CPU time (s) 2.48
Current children cumulated vsize (KiB) 432072
heavy processes:

[startup+6.30167 s]
/proc/loadavg: 8.10 8.08 5.39 9/489 22150
/proc/meminfo: memFree=24532748/32951124 swapFree=60609076/67111528
[pid=22121] ppid=22119 vsize=432072 CPUtime=2.48 cores=0-7
/proc/22121/stat : 22121 (python) S 22119 22121 21743 0 -1 4202496 19880 932 1 0 238 10 0 0 19 0 9 0 668194715 442441728 8766 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624937419 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/22121/statm: 108018 8766 1891 364 0 43915 0
[pid=22127] ppid=22121 vsize=284984 CPUtime=0 cores=0-7
/proc/22127/stat : 22127 (python) S 22121 22121 21743 0 -1 4202560 729 0 0 0 0 0 0 0 18 0 1 0 668194952 291823616 6899 33554432000 4194304 5683644 140735217443040 18446744073709551615 270613130767 0 0 16781312 514 14522477951942885633 0 0 17 1 0 0 0
/proc/22127/statm: 71246 6899 355 364 0 7164 0
[pid=22129] ppid=22127 vsize=1020308 CPUtime=3.91 cores=0-7
/proc/22129/stat : 22129 (gnovelty+2) R 22127 22129 22129 0 -1 4202496 1158 0 0 0 391 0 0 0 25 0 1 0 668194952 1044795392 891 33554432000 134512640 135663704 4287974784 18446744073709551615 134524561 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/22129/statm: 255077 891 113 282 0 254793 0
[pid=22130] ppid=22121 vsize=295004 CPUtime=0 cores=0-7
/proc/22130/stat : 22130 (python) S 22121 22121 21743 0 -1 4202560 737 0 0 0 0 0 0 0 18 0 1 0 668194955 302084096 7032 33554432000 4194304 5683644 140735217443040 18446744073709551615 270613130822 0 0 16781312 514 15190625054741201155 0 0 17 1 0 0 0
/proc/22130/statm: 73751 7032 349 364 0 9666 0
[pid=22132] ppid=22130 vsize=1020308 CPUtime=3.89 cores=0-7
/proc/22132/stat : 22132 (gnovelty+2) R 22130 22132 22132 0 -1 4202496 1153 0 0 0 389 0 0 0 25 0 1 0 668194955 1044795392 891 33554432000 134512640 135663704 4287151136 18446744073709551615 134525258 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/22132/statm: 255077 891 113 282 0 254793 0
[pid=22133] ppid=22121 vsize=370548 CPUtime=0 cores=0-7
/proc/22133/stat : 22133 (python) S 22121 22121 21743 0 -1 4202560 701 0 0 0 0 0 0 0 18 0 1 0 668194958 379441152 7207 33554432000 4194304 5683644 140735217443040 18446744073709551615 270613130822 0 0 16781312 514 11351676632948506882 0 0 17 1 0 0 0
/proc/22133/statm: 92637 7207 348 364 0 28549 0
[pid=22135] ppid=22133 vsize=423400 CPUtime=3.85 cores=0-7
/proc/22135/stat : 22135 (adaptg2wsat2009) R 22133 22135 22135 0 -1 4202496 2610 0 0 0 385 0 0 0 25 0 1 0 668194958 433561600 2356 33554432000 134512640 135107411 4291623344 18446744073709551615 134532277 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/22135/statm: 105850 2356 59 146 0 104726 0
[pid=22136] ppid=22121 vsize=380804 CPUtime=0 cores=0-7
/proc/22136/stat : 22136 (python) S 22121 22121 21743 0 -1 4202560 713 0 0 0 0 0 0 0 18 0 1 0 668194961 389943296 7168 33554432000 4194304 5683644 140735217443040 18446744073709551615 270613130822 0 0 16781312 514 11799681716656767235 0 0 17 1 0 0 0
/proc/22136/statm: 95201 7168 348 364 0 31110 0
[pid=22138] ppid=22136 vsize=423400 CPUtime=3.83 cores=0-7
/proc/22138/stat : 22138 (adaptg2wsat2009) R 22136 22138 22138 0 -1 4202496 2613 0 0 0 383 0 0 0 25 0 1 0 668194961 433561600 2357 33554432000 134512640 135107411 4289865760 18446744073709551615 134532128 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/22138/statm: 105850 2357 59 146 0 104726 0
[pid=22139] ppid=22121 vsize=391060 CPUtime=0 cores=0-7
/proc/22139/stat : 22139 (python) S 22121 22121 21743 0 -1 4202560 721 0 0 0 0 0 0 0 18 0 1 0 668194964 400445440 7173 33554432000 4194304 5683644 140735217443040 18446744073709551615 270613130822 0 0 16781312 514 4641330780352512257 0 0 17 1 0 0 0
/proc/22139/statm: 97765 7173 348 364 0 33671 0
[pid=22141] ppid=22139 vsize=423400 CPUtime=3.72 cores=0-7
/proc/22141/stat : 22141 (adaptg2wsat2009) R 22139 22141 22141 0 -1 4202496 2613 0 0 0 372 0 0 0 25 0 1 0 668194964 433561600 2356 33554432000 134512640 135107411 4294630032 18446744073709551615 134531600 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/22141/statm: 105850 2356 59 146 0 104726 0
[pid=22142] ppid=22121 vsize=401316 CPUtime=0 cores=0-7
/proc/22142/stat : 22142 (python) S 22121 22121 21743 0 -1 4202560 730 0 0 0 0 0 0 0 18 0 1 0 668194967 410947584 7194 33554432000 4194304 5683644 140735217443040 18446744073709551615 270613130822 0 0 16781312 514 1157905045354676481 0 0 17 4 0 0 0
/proc/22142/statm: 100329 7194 348 364 0 36232 0
[pid=22144] ppid=22142 vsize=1020308 CPUtime=3.76 cores=0-7
/proc/22144/stat : 22144 (gnovelty+2) R 22142 22144 22144 0 -1 4202496 1149 0 0 0 376 0 0 0 25 0 1 0 668194967 1044795392 891 33554432000 134512640 135663704 4291277904 18446744073709551615 134545631 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/22144/statm: 255077 891 113 282 0 254793 0
[pid=22145] ppid=22121 vsize=411572 CPUtime=0 cores=0-7
/proc/22145/stat : 22145 (python) S 22121 22121 21743 0 -1 4202560 732 0 0 0 0 0 0 0 18 0 1 0 668194970 421449728 7198 33554432000 4194304 5683644 140735217443040 18446744073709551615 270613130822 0 0 16781312 514 15779069934031569155 0 0 17 1 0 0 0
/proc/22145/statm: 102893 7198 348 364 0 38793 0
[pid=22147] ppid=22145 vsize=253028 CPUtime=3.69 cores=0-7
/proc/22147/stat : 22147 (FH) R 22145 22147 22147 0 -1 4202496 2493 0 0 0 369 0 0 0 25 0 1 0 668194973 259100672 2233 33554432000 134512640 134936988 4288159568 18446744073709551615 134545320 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/22147/statm: 63257 2233 41 104 0 62761 0
[pid=22148] ppid=22121 vsize=421828 CPUtime=0 cores=0-7
/proc/22148/stat : 22148 (python) S 22121 22121 21743 0 -1 4202560 766 0 0 0 0 0 0 0 18 0 1 0 668194974 431951872 7225 33554432000 4194304 5683644 140735217443040 18446744073709551615 270613130822 0 0 16781312 514 6119508165420613889 0 0 17 1 0 0 0
/proc/22148/statm: 105457 7225 354 364 0 41354 0
[pid=22150] ppid=22148 vsize=423400 CPUtime=3.68 cores=0-7
/proc/22150/stat : 22150 (adaptg2wsat2009) R 22148 22150 22150 0 -1 4202496 2622 0 0 0 368 0 0 0 25 0 1 0 668194975 433561600 2357 33554432000 134512640 135107411 4286771792 18446744073709551615 134531215 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/22150/statm: 105850 2357 59 146 0 104726 0
[pid=22121/tid=22128] ppid=22119 vsize=432072 CPUtime=0 cores=0-7
/proc/22121/task/22128/stat : 22128 (python) S 22119 22121 21743 0 -1 4202560 25 932 0 0 0 0 0 0 21 0 9 0 668194952 442441728 8766 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 0 0 0 0
[pid=22121/tid=22131] ppid=22119 vsize=432072 CPUtime=0 cores=0-7
/proc/22121/task/22131/stat : 22131 (python) S 22119 22121 21743 0 -1 4202560 25 932 0 0 0 0 0 0 22 0 9 0 668194955 442441728 8766 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 3 0 0 0
[pid=22121/tid=22134] ppid=22119 vsize=432072 CPUtime=0 cores=0-7
/proc/22121/task/22134/stat : 22134 (python) S 22119 22121 21743 0 -1 4202560 88 932 0 0 0 0 0 0 19 0 9 0 668194958 442441728 8766 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 0 0 0 0
[pid=22121/tid=22137] ppid=22119 vsize=432072 CPUtime=0 cores=0-7
/proc/22121/task/22137/stat : 22137 (python) S 22119 22121 21743 0 -1 4202560 31 932 0 0 0 0 0 0 18 0 9 0 668194961 442441728 8766 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 5 0 0 0
[pid=22121/tid=22140] ppid=22119 vsize=432072 CPUtime=0 cores=0-7
/proc/22121/task/22140/stat : 22140 (python) S 22119 22121 21743 0 -1 4202560 37 932 0 0 0 0 0 0 21 0 9 0 668194964 442441728 8766 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 0 0 0 0
[pid=22121/tid=22143] ppid=22119 vsize=432072 CPUtime=0 cores=0-7
/proc/22121/task/22143/stat : 22143 (python) S 22119 22121 21743 0 -1 4202560 34 932 0 0 0 0 0 0 23 0 9 0 668194967 442441728 8766 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 4 0 0 0
[pid=22121/tid=22146] ppid=22119 vsize=432072 CPUtime=0 cores=0-7
/proc/22121/task/22146/stat : 22146 (python) S 22119 22121 21743 0 -1 4202560 35 932 0 0 0 0 0 0 19 0 9 0 668194971 442441728 8766 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 0 0 0 0
[pid=22121/tid=22149] ppid=22119 vsize=432072 CPUtime=0 cores=0-7
/proc/22121/task/22149/stat : 22149 (python) S 22119 22121 21743 0 -1 4202560 33 932 0 0 0 0 0 0 20 0 9 0 668194974 442441728 8766 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 32.81
Current children cumulated vsize (KiB) 8396740

[startup+12.7015 s]
/proc/loadavg: 8.09 8.08 5.40 9/489 22150
/proc/meminfo: memFree=24532416/32951124 swapFree=60609076/67111528
[pid=22121] ppid=22119 vsize=432072 CPUtime=2.48 cores=0-7
/proc/22121/stat : 22121 (python) S 22119 22121 21743 0 -1 4202496 19880 932 1 0 238 10 0 0 19 0 9 0 668194715 442441728 8766 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624937419 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/22121/statm: 108018 8766 1891 364 0 43915 0
[pid=22127] ppid=22121 vsize=284984 CPUtime=0.04 cores=0-7
/proc/22127/stat : 22127 (python) S 22121 22121 21743 0 -1 4202560 896 0 0 0 2 2 0 0 18 0 1 0 668194952 291823616 6922 33554432000 4194304 5683644 140735217443040 18446744073709551615 270613130767 0 0 16781312 514 10019752706899411201 0 0 17 1 0 0 0
/proc/22127/statm: 71246 6922 378 364 0 7164 0
[pid=22129] ppid=22127 vsize=1020308 CPUtime=10.31 cores=0-7
/proc/22129/stat : 22129 (gnovelty+2) R 22127 22129 22129 0 -1 4202496 1158 0 0 0 1031 0 0 0 25 0 1 0 668194952 1044795392 891 33554432000 134512640 135663704 4287974784 18446744073709551615 134524918 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/22129/statm: 255077 891 113 282 0 254793 0
[pid=22130] ppid=22121 vsize=295004 CPUtime=0.03 cores=0-7
/proc/22130/stat : 22130 (python) S 22121 22121 21743 0 -1 4202560 915 0 0 0 1 2 0 0 18 0 1 0 668194955 302084096 7055 33554432000 4194304 5683644 140735217443040 18446744073709551615 270613130822 0 0 16781312 514 10019752706899411203 0 0 17 1 0 0 0
/proc/22130/statm: 73751 7055 372 364 0 9666 0
[pid=22132] ppid=22130 vsize=1020308 CPUtime=10.21 cores=0-7
/proc/22132/stat : 22132 (gnovelty+2) R 22130 22132 22132 0 -1 4202496 1153 0 0 0 1021 0 0 0 25 0 1 0 668194955 1044795392 891 33554432000 134512640 135663704 4287151136 18446744073709551615 134546070 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/22132/statm: 255077 891 113 282 0 254793 0
[pid=22133] ppid=22121 vsize=370548 CPUtime=0.04 cores=0-7
/proc/22133/stat : 22133 (python) S 22121 22121 21743 0 -1 4202560 782 0 0 0 2 2 0 0 18 0 1 0 668194958 379441152 7230 33554432000 4194304 5683644 140735217443040 18446744073709551615 270613130822 0 0 16781312 514 10019752706899411202 0 0 17 1 0 0 0
/proc/22133/statm: 92637 7230 371 364 0 28549 0
[pid=22135] ppid=22133 vsize=423400 CPUtime=10.25 cores=0-7
/proc/22135/stat : 22135 (adaptg2wsat2009) R 22133 22135 22135 0 -1 4202496 2610 0 0 0 1025 0 0 0 25 0 1 0 668194958 433561600 2356 33554432000 134512640 135107411 4291623344 18446744073709551615 134531781 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/22135/statm: 105850 2356 59 146 0 104726 0
[pid=22136] ppid=22121 vsize=380804 CPUtime=0.03 cores=0-7
/proc/22136/stat : 22136 (python) S 22121 22121 21743 0 -1 4202560 794 0 0 0 2 1 0 0 18 0 1 0 668194961 389943296 7191 33554432000 4194304 5683644 140735217443040 18446744073709551615 270613130822 0 0 16781312 514 10019752706899411203 0 0 17 3 0 0 0
/proc/22136/statm: 95201 7191 371 364 0 31110 0
[pid=22138] ppid=22136 vsize=423400 CPUtime=10.23 cores=0-7
/proc/22138/stat : 22138 (adaptg2wsat2009) R 22136 22138 22138 0 -1 4202496 2613 0 0 0 1023 0 0 0 25 0 1 0 668194961 433561600 2357 33554432000 134512640 135107411 4289865760 18446744073709551615 134539260 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/22138/statm: 105850 2357 59 146 0 104726 0
[pid=22139] ppid=22121 vsize=391060 CPUtime=0.04 cores=0-7
/proc/22139/stat : 22139 (python) S 22121 22121 21743 0 -1 4202560 805 0 0 0 2 2 0 0 18 0 1 0 668194964 400445440 7196 33554432000 4194304 5683644 140735217443040 18446744073709551615 270613130822 0 0 16781312 514 10019752706899411201 0 0 17 3 0 0 0
/proc/22139/statm: 97765 7196 371 364 0 33671 0
[pid=22141] ppid=22139 vsize=423400 CPUtime=10.01 cores=0-7
/proc/22141/stat : 22141 (adaptg2wsat2009) R 22139 22141 22141 0 -1 4202496 2613 0 0 0 1001 0 0 0 25 0 1 0 668194964 433561600 2356 33554432000 134512640 135107411 4294630032 18446744073709551615 134531456 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/22141/statm: 105850 2356 59 146 0 104726 0
[pid=22142] ppid=22121 vsize=401316 CPUtime=0.03 cores=0-7
/proc/22142/stat : 22142 (python) S 22121 22121 21743 0 -1 4202560 814 0 0 0 2 1 0 0 18 0 1 0 668194967 410947584 7217 33554432000 4194304 5683644 140735217443040 18446744073709551615 270613130822 0 0 16781312 514 10019752706899411201 0 0 17 4 0 0 0
/proc/22142/statm: 100329 7217 371 364 0 36232 0
[pid=22144] ppid=22142 vsize=1020308 CPUtime=10.12 cores=0-7
/proc/22144/stat : 22144 (gnovelty+2) R 22142 22144 22144 0 -1 4202496 1149 0 0 0 1012 0 0 0 25 0 1 0 668194967 1044795392 891 33554432000 134512640 135663704 4291277904 18446744073709551615 134525678 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/22144/statm: 255077 891 113 282 0 254793 0
[pid=22145] ppid=22121 vsize=411572 CPUtime=0.04 cores=0-7
/proc/22145/stat : 22145 (python) S 22121 22121 21743 0 -1 4202560 821 0 0 0 2 2 0 0 18 0 1 0 668194970 421449728 7221 33554432000 4194304 5683644 140735217443040 18446744073709551615 270613130822 0 0 16781312 514 10019752706899411203 0 0 17 1 0 0 0

################
# More data... #
################

/proc/22213/stat : 22213 (TNM) R 22211 22213 22213 0 -1 4202496 2469 0 0 0 79555 3 0 0 25 0 1 0 668234253 472580096 2194 33554432000 134512640 135008748 4294792320 18446744073709551615 134527750 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/22213/statm: 115376 2194 46 122 0 114276 0
[pid=22215] ppid=22121 vsize=555156 CPUtime=0.38 cores=0-7
/proc/22215/stat : 22215 (python) S 22121 22121 21743 0 -1 4202560 1160 0 0 0 20 18 0 0 19 0 2 0 668234831 568479744 7633 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/22215/statm: 138789 7633 400 364 0 74647 0
[pid=22217] ppid=22215 vsize=335996 CPUtime=99.4 cores=0-7
/proc/22217/stat : 22217 (iPAWS) T 22215 22217 22217 0 -1 4202496 2048 0 0 0 9940 0 0 0 25 0 1 0 668234831 344059904 1773 33554432000 134512640 135114423 4288596768 18446744073709551615 134517504 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/22217/statm: 83999 1773 59 147 0 83851 0
[pid=22215/tid=22223] ppid=22121 vsize=555156 CPUtime=0 cores=0-7
/proc/22215/task/22223/stat : 22223 (python) S 22121 22121 21743 0 -1 4202560 88 0 0 0 0 0 0 0 19 0 2 0 668244772 568479744 7633 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 514 18446604441569043392 0 0 -1 3 0 0 0
[pid=22220] ppid=22121 vsize=565412 CPUtime=0.38 cores=0-7
/proc/22220/stat : 22220 (python) S 22121 22121 21743 0 -1 4202560 1180 0 0 0 21 17 0 0 19 0 2 0 668244093 578981888 7639 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/22220/statm: 141353 7639 406 364 0 77208 0
[pid=22222] ppid=22220 vsize=1020308 CPUtime=100.46 cores=0-7
/proc/22222/stat : 22222 (gnovelty+2) T 22220 22222 22222 0 -1 4202496 1176 0 0 0 10046 0 0 0 25 0 1 0 668244094 1044795392 892 33554432000 134512640 135663704 4290135296 18446744073709551615 134545607 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/22222/statm: 255077 892 113 282 0 254793 0
[pid=22220/tid=22231] ppid=22121 vsize=565412 CPUtime=0 cores=0-7
/proc/22220/task/22231/stat : 22231 (python) S 22121 22121 21743 0 -1 4202560 84 0 0 0 0 0 0 0 19 0 2 0 668254141 578981888 7639 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 514 18446604441452597696 0 0 -1 5 0 0 0
[pid=22228] ppid=22121 vsize=575668 CPUtime=2.39 cores=0-7
/proc/22228/stat : 22228 (python) S 22121 22121 21743 0 -1 4202560 1217 0 0 0 106 133 0 0 18 0 2 0 668253924 589484032 7624 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/22228/statm: 143917 7624 406 364 0 79769 0
[pid=22230] ppid=22228 vsize=268424 CPUtime=586.54 cores=0-7
/proc/22230/stat : 22230 (hybridGM3) T 22228 22230 22230 0 -1 4202496 6201 0 0 0 58653 1 0 0 25 0 1 0 668253924 274866176 5918 33554432000 134512640 135204385 4287855920 18446744073709551615 134538475 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/22230/statm: 67106 5918 63 169 0 66936 0
[pid=22228/tid=22246] ppid=22121 vsize=575668 CPUtime=0 cores=0-7
/proc/22228/task/22246/stat : 22246 (python) S 22121 22121 21743 0 -1 4202560 91 0 0 0 0 0 0 0 18 0 2 0 668273518 589484032 7624 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 514 18446604467742496576 0 0 -1 1 0 0 0
[pid=22232] ppid=22121 vsize=585924 CPUtime=1.19 cores=0-7
/proc/22232/stat : 22232 (python) S 22121 22121 21743 0 -1 4202560 1209 0 0 0 58 61 0 0 18 0 2 0 668254147 599986176 7676 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/22232/statm: 146481 7676 407 364 0 82330 0
[pid=22234] ppid=22232 vsize=268424 CPUtime=293.52 cores=0-7
/proc/22234/stat : 22234 (hybridGM3) T 22232 22234 22234 0 -1 4202496 6203 0 0 0 29351 1 0 0 25 0 1 0 668254147 274866176 5918 33554432000 134512640 135204385 4288617232 18446744073709551615 134538041 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/22234/statm: 67106 5918 63 169 0 66936 0
[pid=22232/tid=22250] ppid=22121 vsize=585924 CPUtime=0 cores=0-7
/proc/22232/task/22250/stat : 22250 (python) S 22121 22121 21743 0 -1 4202560 87 0 0 0 0 0 0 0 18 0 2 0 668273638 599986176 7676 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 514 18446604465607209984 0 0 -1 2 0 0 0
[pid=22235] ppid=22121 vsize=596180 CPUtime=1.19 cores=0-7
/proc/22235/stat : 22235 (python) S 22121 22121 21743 0 -1 4202560 1199 0 0 0 58 61 0 0 18 0 2 0 668254247 610488320 7690 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/22235/statm: 149045 7690 400 364 0 84891 0
[pid=22237] ppid=22235 vsize=268424 CPUtime=292.05 cores=0-7
/proc/22237/stat : 22237 (hybridGM3) T 22235 22237 22237 0 -1 4202496 6202 0 0 0 29204 1 0 0 25 0 1 0 668254247 274866176 5919 33554432000 134512640 135204385 4291604528 18446744073709551615 134537699 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/22237/statm: 67106 5919 63 169 0 66936 0
[pid=22235/tid=22251] ppid=22121 vsize=596180 CPUtime=0 cores=0-7
/proc/22235/task/22251/stat : 22251 (python) S 22121 22121 21743 0 -1 4202560 86 0 0 0 0 0 0 0 18 0 2 0 668273838 610488320 7690 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 514 18446604469307564480 0 0 -1 6 0 0 0
[pid=22238] ppid=22121 vsize=606436 CPUtime=2.35 cores=0-7
/proc/22238/stat : 22238 (python) S 22121 22121 21743 0 -1 4202560 1002 0 0 0 107 128 0 0 18 0 1 0 668254522 620990464 7625 33554432000 4194304 5683644 140735217443040 18446744073709551615 270613130822 0 0 16781312 514 10019752706899411205 0 0 17 3 0 0 0
/proc/22238/statm: 151609 7625 372 364 0 87452 0
[pid=22240] ppid=22238 vsize=461504 CPUtime=591.79 cores=0-7
/proc/22240/stat : 22240 (TNM) R 22238 22240 22240 0 -1 4202496 2474 0 0 0 59178 1 0 0 25 0 1 0 668254523 472580096 2194 33554432000 134512640 135008748 4291887680 18446744073709551615 134529448 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/22240/statm: 115376 2194 46 122 0 114276 0
[pid=22247] ppid=22121 vsize=616692 CPUtime=0.4 cores=0-7
/proc/22247/stat : 22247 (python) S 22121 22121 21743 0 -1 4202560 1196 0 0 0 21 19 0 0 18 0 2 0 668273524 631492608 7626 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/22247/statm: 154173 7626 401 364 0 90013 0
[pid=22249] ppid=22247 vsize=335996 CPUtime=100.38 cores=0-7
/proc/22249/stat : 22249 (iPAWS) T 22247 22249 22249 0 -1 4202496 2059 0 0 0 10038 0 0 0 25 0 1 0 668273525 344059904 1776 33554432000 134512640 135114423 4287485152 18446744073709551615 134547175 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/22249/statm: 83999 1776 59 147 0 83851 0
[pid=22247/tid=22255] ppid=22121 vsize=616692 CPUtime=0 cores=0-7
/proc/22247/task/22255/stat : 22255 (python) S 22121 22121 21743 0 -1 4202560 89 0 0 0 0 0 0 0 19 0 2 0 668283564 631492608 7626 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 514 18446604456466032512 0 0 -1 0 0 0 0
[pid=22258] ppid=22121 vsize=626948 CPUtime=0.84 cores=0-7
/proc/22258/stat : 22258 (python) S 22121 22121 21743 0 -1 4202560 1221 0 0 0 38 46 0 0 18 0 2 0 668293315 641994752 7612 33554432000 4194304 5683644 140735217443040 18446744073709551615 270613130822 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/22258/statm: 156737 7612 400 364 0 92574 0
[pid=22260] ppid=22258 vsize=253036 CPUtime=203.1 cores=0-7
/proc/22260/stat : 22260 (FH) R 22258 22260 22260 0 -1 4202496 2521 0 0 0 20310 0 0 0 25 0 1 0 668293316 259108864 2233 33554432000 134512640 134936988 4294114080 18446744073709551615 134553582 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/22260/statm: 63259 2233 41 104 0 62763 0
[pid=22258/tid=22267] ppid=22121 vsize=626948 CPUtime=0 cores=0-7
/proc/22258/task/22267/stat : 22267 (python) S 22121 22121 21743 0 -1 4202560 93 0 0 0 0 0 0 0 19 0 2 0 668303957 641994752 7612 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 514 18446604456380258240 0 0 -1 7 0 0 0
[pid=22261] ppid=22121 vsize=637204 CPUtime=0.81 cores=0-7
/proc/22261/stat : 22261 (python) S 22121 22121 21743 0 -1 4202560 1055 0 0 0 40 41 0 0 18 0 1 0 668293809 652496896 7674 33554432000 4194304 5683644 140735217443040 18446744073709551615 270613130822 0 0 16781312 514 10019752706899411206 0 0 17 3 0 0 0
/proc/22261/statm: 159301 7674 379 364 0 95135 0
[pid=22263] ppid=22261 vsize=423312 CPUtime=203.51 cores=0-7
/proc/22263/stat : 22263 (NCVWr) R 22261 22263 22263 0 -1 4202496 2562 0 0 0 20351 0 0 0 25 0 1 0 668293810 433471488 2266 33554432000 134512640 135113558 4287332416 18446744073709551615 134531797 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/22263/statm: 105828 2266 58 147 0 104703 0
[pid=22268] ppid=22121 vsize=647460 CPUtime=0.4 cores=0-7
/proc/22268/stat : 22268 (python) S 22121 22121 21743 0 -1 4202560 1254 0 0 0 19 21 0 0 19 0 2 0 668304019 662999040 7634 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/22268/statm: 161865 7634 407 364 0 97696 0
[pid=22270] ppid=22268 vsize=1020308 CPUtime=99.06 cores=0-7
/proc/22270/stat : 22270 (gnovelty+2) T 22268 22270 22270 0 -1 4202496 1180 0 0 0 9906 0 0 0 25 0 1 0 668304019 1044795392 892 33554432000 134512640 135663704 4292731568 18446744073709551615 134525224 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/22270/statm: 255077 892 113 282 0 254793 0
[pid=22268/tid=22277] ppid=22121 vsize=647460 CPUtime=0 cores=0-7
/proc/22268/task/22277/stat : 22277 (python) S 22121 22121 21743 0 -1 4202560 90 0 0 0 0 0 0 0 19 0 2 0 668314064 662999040 7634 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 514 18446604469672573056 0 0 -1 6 0 0 0
[pid=22274] ppid=22121 vsize=657716 CPUtime=0.01 cores=0-7
/proc/22274/stat : 22274 (python) S 22121 22121 21743 0 -1 4202560 1073 0 0 0 0 1 0 0 18 0 1 0 668313417 673501184 7619 33554432000 4194304 5683644 140735217443040 18446744073709551615 270613130822 0 0 16781312 514 11153973996913918213 0 0 17 3 0 0 0
/proc/22274/statm: 164429 7619 379 364 0 100257 0
[pid=22276] ppid=22274 vsize=268420 CPUtime=7.38 cores=0-7
/proc/22276/stat : 22276 (hybridGM3) R 22274 22276 22276 0 -1 4202496 6208 0 0 0 738 0 0 0 25 0 1 0 668313418 274862080 5919 33554432000 134512640 135204385 4290909472 18446744073709551615 134538395 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/22276/statm: 67105 5919 63 169 0 66935 0
[pid=22278] ppid=22121 vsize=667972 CPUtime=0 cores=0-7
/proc/22278/stat : 22278 (python) S 22121 22121 21743 0 -1 4202560 938 0 0 0 0 0 0 0 19 0 1 0 668314070 684003328 7693 33554432000 4194304 5683644 140735217443040 18446744073709551615 270613130822 0 0 16781312 514 14058769368288854015 0 0 17 3 0 0 0
/proc/22278/statm: 166993 7693 352 364 0 102818 0
[pid=22280] ppid=22278 vsize=335996 CPUtime=0.95 cores=0-7
/proc/22280/stat : 22280 (iPAWS) R 22278 22280 22280 0 -1 4202496 2061 0 0 0 95 0 0 0 23 0 1 0 668314071 344059904 1773 33554432000 134512640 135114423 4290254512 18446744073709551615 134519414 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/22280/statm: 83999 1773 59 147 0 83851 0
[pid=22121/tid=22128] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22128/stat : 22128 (python) S 22119 22121 21743 0 -1 4202560 25 932 0 0 0 0 0 0 21 0 33 0 668194952 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 0 0 0 0
[pid=22121/tid=22131] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22131/stat : 22131 (python) S 22119 22121 21743 0 -1 4202560 25 932 0 0 0 0 0 0 22 0 33 0 668194955 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 3 0 0 0
[pid=22121/tid=22134] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22134/stat : 22134 (python) S 22119 22121 21743 0 -1 4202560 94 932 0 0 0 0 0 0 18 0 33 0 668194958 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 0 0 0 0
[pid=22121/tid=22137] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22137/stat : 22137 (python) S 22119 22121 21743 0 -1 4202560 35 932 0 0 0 0 0 0 18 0 33 0 668194961 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 5 0 0 0
[pid=22121/tid=22140] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22140/stat : 22140 (python) S 22119 22121 21743 0 -1 4202560 41 932 0 0 0 0 0 0 18 0 33 0 668194964 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 0 0 0 0
[pid=22121/tid=22143] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22143/stat : 22143 (python) S 22119 22121 21743 0 -1 4202560 34 932 0 0 0 0 0 0 23 0 33 0 668194967 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 4 0 0 0
[pid=22121/tid=22146] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22146/stat : 22146 (python) S 22119 22121 21743 0 -1 4202560 41 932 0 0 0 0 0 0 18 0 33 0 668194971 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 0 0 0 0
[pid=22121/tid=22149] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22149/stat : 22149 (python) S 22119 22121 21743 0 -1 4202560 48 932 0 0 0 0 0 0 18 0 33 0 668194974 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 1 0 0 0
[pid=22121/tid=22156] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22156/stat : 22156 (python) S 22119 22121 21743 0 -1 4202560 47 932 0 0 0 0 0 0 18 0 33 0 668204920 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 0 0 0 0
[pid=22121/tid=22161] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22161/stat : 22161 (python) S 22119 22121 21743 0 -1 4202560 47 932 0 0 0 0 0 0 18 0 33 0 668204999 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 1 0 0 0
[pid=22121/tid=22166] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22166/stat : 22166 (python) S 22119 22121 21743 0 -1 4202560 39 932 0 0 0 0 0 0 15 0 33 0 668205003 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 7 0 0 0
[pid=22121/tid=22170] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22170/stat : 22170 (python) S 22119 22121 21743 0 -1 4202560 34 932 0 0 0 0 0 0 22 0 33 0 668205008 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 7 0 0 0
[pid=22121/tid=22174] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22174/stat : 22174 (python) S 22119 22121 21743 0 -1 4202560 50 932 0 0 0 0 0 0 18 0 33 0 668205012 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 2 0 0 0
[pid=22121/tid=22178] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22178/stat : 22178 (python) S 22119 22121 21743 0 -1 4202560 55 932 0 0 0 0 0 0 18 0 33 0 668205016 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 2 0 0 0
[pid=22121/tid=22181] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22181/stat : 22181 (python) S 22119 22121 21743 0 -1 4202560 55 932 0 0 0 0 0 0 18 0 33 0 668205021 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 5 0 0 0
[pid=22121/tid=22184] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22184/stat : 22184 (python) S 22119 22121 21743 0 -1 4202560 39 932 0 0 0 0 0 0 18 0 33 0 668205026 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 4 0 0 0
[pid=22121/tid=22190] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22190/stat : 22190 (python) S 22119 22121 21743 0 -1 4202560 32 932 0 0 0 0 0 0 20 0 33 0 668214962 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 3 0 0 0
[pid=22121/tid=22194] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22194/stat : 22194 (python) S 22119 22121 21743 0 -1 4202560 34 932 0 0 0 0 0 0 20 0 33 0 668215045 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 4 0 0 0
[pid=22121/tid=22207] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22207/stat : 22207 (python) S 22119 22121 21743 0 -1 4202560 60 932 0 0 0 0 0 0 18 0 33 0 668225208 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 1 0 0 0
[pid=22121/tid=22212] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22212/stat : 22212 (python) S 22119 22121 21743 0 -1 4202560 36 932 0 0 0 0 0 0 19 0 33 0 668234253 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 4 0 0 0
[pid=22121/tid=22216] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22216/stat : 22216 (python) S 22119 22121 21743 0 -1 4202560 42 932 0 0 0 0 0 0 20 0 33 0 668234831 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 4 0 0 0
[pid=22121/tid=22221] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22221/stat : 22221 (python) S 22119 22121 21743 0 -1 4202560 37 932 0 0 0 0 0 0 18 0 33 0 668244093 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 3 0 0 0
[pid=22121/tid=22229] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22229/stat : 22229 (python) S 22119 22121 21743 0 -1 4202560 52 932 0 0 0 0 0 0 18 0 33 0 668253924 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 0 0 0 0
[pid=22121/tid=22233] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22233/stat : 22233 (python) S 22119 22121 21743 0 -1 4202560 35 932 0 0 0 0 0 0 18 0 33 0 668254147 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 3 0 0 0
[pid=22121/tid=22236] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22236/stat : 22236 (python) S 22119 22121 21743 0 -1 4202560 35 932 0 0 0 0 0 0 18 0 33 0 668254247 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 3 0 0 0
[pid=22121/tid=22239] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22239/stat : 22239 (python) S 22119 22121 21743 0 -1 4202560 31 932 0 0 0 0 0 0 20 0 33 0 668254522 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 3 0 0 0
[pid=22121/tid=22248] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22248/stat : 22248 (python) S 22119 22121 21743 0 -1 4202560 36 932 0 0 0 0 0 0 20 0 33 0 668273525 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 3 0 0 0
[pid=22121/tid=22259] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22259/stat : 22259 (python) S 22119 22121 21743 0 -1 4202560 42 932 0 0 0 0 0 0 18 0 33 0 668293315 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 3 0 0 0
[pid=22121/tid=22262] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22262/stat : 22262 (python) S 22119 22121 21743 0 -1 4202560 36 932 0 0 0 0 0 0 20 0 33 0 668293810 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 3 0 0 0
[pid=22121/tid=22269] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22269/stat : 22269 (python) S 22119 22121 21743 0 -1 4202560 34 932 0 0 0 0 0 0 20 0 33 0 668304019 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 0 0 0 0
[pid=22121/tid=22275] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22275/stat : 22275 (python) S 22119 22121 21743 0 -1 4202560 36 932 0 0 0 0 0 0 20 0 33 0 668313417 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 0 0 0 0
[pid=22121/tid=22279] ppid=22119 vsize=678216 CPUtime=0 cores=0-7
/proc/22121/task/22279/stat : 22279 (python) S 22119 22121 21743 0 -1 4202560 32 932 0 0 0 0 0 0 20 0 33 0 668314071 694493184 9285 33554432000 4194304 5683644 140735217443040 18446744073709551615 270624934321 0 0 16781312 2 18446604446737253632 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 9490.66
Current children cumulated vsize (KiB) 31752248

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 22121 and gives
#  childrusage.ru_utime.tv_sec=2879
#  childrusage.ru_utime.tv_usec=325276
#  childrusage.ru_stime.tv_sec=17
#  childrusage.ru_stime.tv_usec=604323
# CPU time returned by wait4() is 2896.93
# while last known CPU time is 9490.66
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1194.72
CPU time (s): 9490.66
CPU user time (s): 9471.86
CPU system time (s): 18.8
CPU usage (%): 794.387
Max. virtual memory (cumulated for all children) (KiB): 31752248

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2879.33
system time used= 17.6043
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 94076
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 12738
involuntary context switches= 19738

runsolver used 13.9429 second user time and 32.97 second system time

The end

Launcher Data 

Begin job on node108 at 2011-05-16 19:12:18
IDJOB=3385371
IDBENCH=82636
IDSOLVER=1870
FILE ID=node108/3385371-1305565937
RUNJOBID= node108-1305564966-21757
PBS_JOBID= 13350142
Free space on /tmp= 73464 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k5-r20-v2000-c40000-S971556279-001-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385371-1305565937/watcher-3385371-1305565937 -o /tmp/evaluation-result-3385371-1305565937/solver-3385371-1305565937 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385371-1305565937.cnf -seed 1655447407 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 9224277e61359574518ca553717e02ee
RANDOM SEED=1655447407

node108.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      24644952 kB
Buffers:         56512 kB
Cached:         711828 kB
SwapCached:      25764 kB
Active:         776568 kB
Inactive:      7311236 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      24644952 kB
SwapTotal:    67111528 kB
SwapFree:     60609076 kB
Dirty:           33464 kB
Writeback:          28 kB
AnonPages:     7302704 kB
Mapped:          26640 kB
Slab:            83300 kB
PageTables:      61020 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 122563772 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73432 MiB
End job on node108 at 2011-05-16 19:32:14