Trace number 3728391

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
wbo 1.72? (TO) 1800.06 1800.52

General information on the benchmark

Name/SOFT-SMALLINT-LIN/PB06/submitted-PB05/aloul/FPGA_SAT05/
normalized-chnl30_40_pb.cnf.cr--soft-0-100-0.wbo
MD5SUM68a136cd2642dd27a3fdbf065e2a5463
Bench CategorySOFT-SMALLINT-LIN (only soft constraints, small integers, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark10
Best CPU time to get the best result obtained on this benchmark0.554914
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables2400
Total number of constraints140
Number of soft constraints140
Number of constraints which are clauses80
Number of constraints which are cardinality constraints (but not clauses)60
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint30
Maximum length of a constraint40
Top cost 7122
Min constraint cost 1
Max constraint cost 99
Sum of constraints costs 7121
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 41
Number of bits of the biggest sum of numbers6
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c ==============================[ Solver Version ]===============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |   wbo:                     1.72 beta - May, 2012                            |
0.00/0.00	c |                                                                             |
0.00/0.00	c Parsing wbo file format. File HOME/instance-3728391-1338675011.wbo.
0.00/0.00	c Instance file HOME/instance-3728391-1338675011.wbo
0.00/0.00	c File size is 42976 bytes.
0.00/0.00	c Highest Coefficient sum: 41
0.00/0.00	c ============================[ Problem Statistics ]=============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Parsing time:          0.01         s                                      |
0.00/0.00	c |  Number Variables:      2540                                                |
0.00/0.00	c |  Number Clauses:        80                                                  |
0.00/0.00	c |  Number Cardinality:    0                                                   |
0.00/0.00	c |  Number PB Constraints: 60                                                  |
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Problem Type:          Optimization                                        |
0.00/0.00	c |  Objective Function:    PB Constraint                                       |
0.00/0.00	c |                                                                             |
0.00/0.00	c ============================[ Search Statistics ]==============================
0.00/0.00	c | Time (s)  | ID |      Bound      |               Shared Clauses             |
0.00/0.00	c |           |    | value      type |  expCls         impCls        Lit/impCl  |
0.00/0.00	c ===============================================================================
1800.06/1800.51	c ===============================================================================
1800.06/1800.51	c Wall Clock Time: 1800.51 s
1800.06/1800.51	s UNKNOWN

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3728391-1338675011/watcher-3728391-1338675011 -o /tmp/evaluation-result-3728391-1338675011/solver-3728391-1338675011 -C 1800 -W 1900 -M 15500 wbo1.72 -file-format=wbo HOME/instance-3728391-1338675011.wbo 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.02 1.01 1.16 3/178 17952
/proc/meminfo: memFree=31751704/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=4064 CPUtime=0 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) R 17950 17952 17352 0 -1 4202496 557 0 0 0 0 0 0 0 20 0 1 0 333629647 4161536 498 33554432000 4194304 5964350 140736480749568 140733519263480 4297174 0 0 4096 16387 0 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 1136 506 169 433 0 700 0

[startup+0.00863602 s]
/proc/loadavg: 1.02 1.01 1.16 3/178 17952
/proc/meminfo: memFree=31751704/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=84424 CPUtime=0 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 1068 0 0 0 0 0 0 0 20 0 2 0 333629647 86450176 1009 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 21106 1009 178 433 0 20670 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 84424

[startup+0.10015 s]
/proc/loadavg: 1.02 1.01 1.16 3/178 17952
/proc/meminfo: memFree=31751704/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=84424 CPUtime=0.09 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 1500 0 0 0 9 0 0 0 20 0 2 0 333629647 86450176 1441 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 21106 1441 193 433 0 20670 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 84424

[startup+0.300112 s]
/proc/loadavg: 1.02 1.01 1.16 3/178 17952
/proc/meminfo: memFree=31751704/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=84424 CPUtime=0.29 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 2021 0 0 0 29 0 0 0 20 0 2 0 333629647 86450176 1962 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 21106 1962 193 433 0 20670 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 84424

[startup+0.700112 s]
/proc/loadavg: 1.02 1.01 1.16 3/178 17952
/proc/meminfo: memFree=31751704/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=84424 CPUtime=0.69 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 2914 0 0 0 69 0 0 0 20 0 2 0 333629647 86450176 2855 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 21106 2855 193 433 0 20670 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 84424

[startup+1.50025 s]
/proc/loadavg: 1.10 1.02 1.17 3/180 17954
/proc/meminfo: memFree=31730088/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=84424 CPUtime=1.48 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 4467 0 0 0 148 0 0 0 20 0 2 0 333629647 86450176 4408 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 21106 4408 193 433 0 20670 0
[pid=17952/tid=17954] ppid=17950 vsize=84424 CPUtime=1.47 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 3563 0 0 0 147 0 0 0 20 0 2 0 333629648 86450176 4408 33554432000 4194304 5964350 140736480749568 140540007499576 4221120 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 84424

[startup+3.10025 s]
/proc/loadavg: 1.10 1.02 1.17 3/180 17954
/proc/meminfo: memFree=31721292/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=84568 CPUtime=3.09 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 5439 0 0 0 308 1 0 0 20 0 2 0 333629647 86597632 5380 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 21142 5380 193 433 0 20706 0
[pid=17952/tid=17954] ppid=17950 vsize=84568 CPUtime=3.07 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 4535 0 0 0 306 1 0 0 20 0 2 0 333629648 86597632 5380 33554432000 4194304 5964350 140736480749568 140540007499768 4400007 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 84568

[startup+6.30026 s]
/proc/loadavg: 1.10 1.02 1.17 3/180 17954
/proc/meminfo: memFree=31705024/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=84640 CPUtime=6.28 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 7988 0 0 0 627 1 0 0 20 0 2 0 333629647 86671360 7929 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 21160 7929 193 433 0 20724 0
[pid=17952/tid=17954] ppid=17950 vsize=84640 CPUtime=6.26 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 7084 0 0 0 625 1 0 0 20 0 2 0 333629648 86671360 7929 33554432000 4194304 5964350 140736480749568 140540007499768 4399840 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 6.28
Current children cumulated vsize (KiB) 84640

[startup+12.7003 s]
/proc/loadavg: 1.24 1.06 1.17 3/180 17954
/proc/meminfo: memFree=31673328/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=84640 CPUtime=12.69 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 11654 0 0 0 1266 3 0 0 20 0 2 0 333629647 86671360 11595 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 21160 11595 193 433 0 20724 0
[pid=17952/tid=17954] ppid=17950 vsize=84640 CPUtime=12.64 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 10750 0 0 0 1261 3 0 0 20 0 2 0 333629648 86671360 11595 33554432000 4194304 5964350 140736480749568 140540007499768 4399976 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 84640

[startup+25.5003 s]
/proc/loadavg: 1.35 1.09 1.18 3/180 17954
/proc/meminfo: memFree=31645668/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=84748 CPUtime=25.48 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 14715 0 0 0 2545 3 0 0 20 0 2 0 333629647 86781952 14656 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 21187 14656 193 433 0 20751 0
[pid=17952/tid=17954] ppid=17950 vsize=84748 CPUtime=25.41 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 13811 0 0 0 2538 3 0 0 20 0 2 0 333629648 86781952 14656 33554432000 4194304 5964350 140736480749568 140540007499576 4400007 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 25.48
Current children cumulated vsize (KiB) 84748

[startup+51.1004 s]
/proc/loadavg: 1.57 1.16 1.20 3/179 17954
/proc/meminfo: memFree=31590164/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=150444 CPUtime=51.07 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 24568 0 0 0 5100 7 0 0 20 0 2 0 333629647 154054656 24734 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 37611 24734 193 433 0 37175 0
[pid=17952/tid=17954] ppid=17950 vsize=150444 CPUtime=50.94 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 23664 0 0 0 5087 7 0 0 20 0 2 0 333629648 154054656 24734 33554432000 4194304 5964350 140736480749568 140540007499816 4399895 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 51.07
Current children cumulated vsize (KiB) 150444

[startup+102.307 s]
/proc/loadavg: 1.83 1.30 1.25 3/179 17954
/proc/meminfo: memFree=31554000/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=150684 CPUtime=102.24 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 30637 0 0 0 10215 9 0 0 20 0 2 0 333629647 154300416 30855 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 37671 30855 193 433 0 37235 0
[pid=17952/tid=17954] ppid=17950 vsize=150684 CPUtime=101.99 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 29733 0 0 0 10189 10 0 0 20 0 2 0 333629648 154300416 30855 33554432000 4194304 5964350 140736480749568 140540007499768 4388148 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 102.24
Current children cumulated vsize (KiB) 150684

[startup+162.3 s]
/proc/loadavg: 1.93 1.42 1.29 3/179 17955
/proc/meminfo: memFree=31510320/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=216364 CPUtime=162.22 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 40468 0 0 0 16209 13 0 0 20 0 2 0 333629647 221556736 41651 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 54091 41651 193 433 0 53655 0
[pid=17952/tid=17954] ppid=17950 vsize=216364 CPUtime=161.82 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 39564 0 0 0 16167 15 0 0 20 0 2 0 333629648 221556736 41651 33554432000 4194304 5964350 140736480749568 140540007499768 4221120 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 162.22
Current children cumulated vsize (KiB) 216364

[startup+222.3 s]
/proc/loadavg: 1.97 1.53 1.33 3/178 17955
/proc/meminfo: memFree=31494108/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=217444 CPUtime=222.19 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 43369 0 0 0 22204 15 0 0 20 0 2 0 333629647 222662656 44673 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 54361 44673 193 433 0 53925 0
[pid=17952/tid=17954] ppid=17950 vsize=217444 CPUtime=221.66 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 42465 0 0 0 22148 18 0 0 20 0 2 0 333629648 222662656 44673 33554432000 4194304 5964350 140736480749568 140540007499768 4400007 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 222.19
Current children cumulated vsize (KiB) 217444

[startup+282.3 s]
/proc/loadavg: 1.99 1.61 1.37 3/179 17956
/proc/meminfo: memFree=31432068/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=284996 CPUtime=282.17 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 56057 0 0 0 28196 21 0 0 20 0 2 0 333629647 291835904 57427 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 71249 57427 193 433 0 70813 0
[pid=17952/tid=17954] ppid=17950 vsize=284996 CPUtime=281.49 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 55153 0 0 0 28124 25 0 0 20 0 2 0 333629648 291835904 57427 33554432000 4194304 5964350 140736480749568 140540007499768 4399989 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 282.17
Current children cumulated vsize (KiB) 284996

[startup+342.3 s]
/proc/loadavg: 1.99 1.68 1.41 3/179 17956
/proc/meminfo: memFree=31417232/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=285500 CPUtime=342.16 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 58949 0 0 0 34194 22 0 0 20 0 2 0 333629647 292352000 60319 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 71375 60319 193 433 0 70939 0
[pid=17952/tid=17954] ppid=17950 vsize=285500 CPUtime=341.34 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 58045 0 0 0 34107 27 0 0 20 0 2 0 333629648 292352000 60319 33554432000 4194304 5964350 140736480749568 140540007499768 4399976 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 342.16
Current children cumulated vsize (KiB) 285500

[startup+402.3 s]
/proc/loadavg: 1.99 1.73 1.44 3/179 17956
/proc/meminfo: memFree=31412484/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=286076 CPUtime=402.15 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 60058 0 0 0 40192 23 0 0 20 0 2 0 333629647 292941824 61482 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 71519 61482 193 433 0 71083 0
[pid=17952/tid=17954] ppid=17950 vsize=286076 CPUtime=401.18 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 59154 0 0 0 40089 29 0 0 20 0 2 0 333629648 292941824 61482 33554432000 4194304 5964350 140736480749568 140540007499816 4221120 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 402.15
Current children cumulated vsize (KiB) 286076

[startup+462.3 s]
/proc/loadavg: 1.99 1.78 1.47 3/179 17957
/proc/meminfo: memFree=31364912/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=355176 CPUtime=462.13 cores=1,3,5,7

################
# More data... #
################

[pid=17952/tid=17954] ppid=17950 vsize=503052 CPUtime=1119.29 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 112000 0 0 0 111860 69 0 0 20 0 2 0 333629648 515125248 114426 33554432000 4194304 5964350 140736480749568 140540007499576 4400023 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1121.99
Current children cumulated vsize (KiB) 503052

[startup+1182.3 s]
/proc/loadavg: 2.00 1.97 1.74 3/179 17962
/proc/meminfo: memFree=31104460/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=503052 CPUtime=1181.98 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 112908 0 0 0 118146 52 0 0 20 0 2 0 333629647 515125248 114430 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 125763 114430 193 433 0 125327 0
[pid=17952/tid=17954] ppid=17950 vsize=503052 CPUtime=1179.14 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 112004 0 0 0 117845 69 0 0 20 0 2 0 333629648 515125248 114430 33554432000 4194304 5964350 140736480749568 140540007499576 4399852 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1181.98
Current children cumulated vsize (KiB) 503052

[startup+1242.3 s]
/proc/loadavg: 2.06 1.99 1.75 3/180 17965
/proc/meminfo: memFree=31104452/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=503052 CPUtime=1241.97 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 112908 0 0 0 124145 52 0 0 20 0 2 0 333629647 515125248 114430 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 125763 114430 193 433 0 125327 0
[pid=17952/tid=17954] ppid=17950 vsize=503052 CPUtime=1238.99 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 112004 0 0 0 123829 70 0 0 20 0 2 0 333629648 515125248 114430 33554432000 4194304 5964350 140736480749568 140540007499576 4381949 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1241.97
Current children cumulated vsize (KiB) 503052

[startup+1302.3 s]
/proc/loadavg: 2.02 1.99 1.77 3/179 17965
/proc/meminfo: memFree=31103556/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=503052 CPUtime=1301.96 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 112919 0 0 0 130144 52 0 0 20 0 2 0 333629647 515125248 114434 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 125763 114434 193 433 0 125327 0
[pid=17952/tid=17954] ppid=17950 vsize=503052 CPUtime=1298.83 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 112015 0 0 0 129812 71 0 0 20 0 2 0 333629648 515125248 114434 33554432000 4194304 5964350 140736480749568 140540007499576 4381458 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1301.96
Current children cumulated vsize (KiB) 503052

[startup+1362.3 s]
/proc/loadavg: 2.00 1.99 1.78 3/179 17966
/proc/meminfo: memFree=31103432/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=503052 CPUtime=1361.94 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 112919 0 0 0 136142 52 0 0 20 0 2 0 333629647 515125248 114434 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 125763 114434 193 433 0 125327 0
[pid=17952/tid=17954] ppid=17950 vsize=503052 CPUtime=1358.67 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 112015 0 0 0 135795 72 0 0 20 0 2 0 333629648 515125248 114434 33554432000 4194304 5964350 140736480749568 140540007499576 4400390 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1361.94
Current children cumulated vsize (KiB) 503052

[startup+1422.31 s]
/proc/loadavg: 2.00 1.99 1.79 3/179 17966
/proc/meminfo: memFree=31101820/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=503052 CPUtime=1421.94 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 112919 0 0 0 142142 52 0 0 20 0 2 0 333629647 515125248 114434 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 125763 114434 193 433 0 125327 0
[pid=17952/tid=17954] ppid=17950 vsize=503052 CPUtime=1418.53 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 112015 0 0 0 141780 73 0 0 20 0 2 0 333629648 515125248 114434 33554432000 4194304 5964350 140736480749568 140540007499576 4400017 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1421.94
Current children cumulated vsize (KiB) 503052

[startup+1482.3 s]
/proc/loadavg: 2.00 1.99 1.80 3/179 17966
/proc/meminfo: memFree=31093388/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=505140 CPUtime=1481.92 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 113334 0 0 0 148140 52 0 0 20 0 2 0 333629647 517263360 114849 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 126285 114849 193 433 0 125849 0
[pid=17952/tid=17954] ppid=17950 vsize=505140 CPUtime=1478.37 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 112430 0 0 0 147763 74 0 0 20 0 2 0 333629648 517263360 114849 33554432000 4194304 5964350 140736480749568 140540007499816 4220672 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1481.92
Current children cumulated vsize (KiB) 505140

[startup+1542.3 s]
/proc/loadavg: 2.00 1.99 1.81 3/179 17966
/proc/meminfo: memFree=31039496/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=575960 CPUtime=1541.91 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 124181 0 0 0 154133 58 0 0 20 0 2 0 333629647 589783040 125696 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 143990 125696 193 433 0 143554 0
[pid=17952/tid=17954] ppid=17950 vsize=575960 CPUtime=1538.22 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 123277 0 0 0 153741 81 0 0 20 0 2 0 333629648 589783040 125696 33554432000 4194304 5964350 140736480749568 140540007499576 4400011 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1541.91
Current children cumulated vsize (KiB) 575960

[startup+1602.3 s]
/proc/loadavg: 2.00 1.99 1.82 3/179 17966
/proc/meminfo: memFree=31017920/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=577508 CPUtime=1601.9 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 129492 0 0 0 160129 61 0 0 20 0 2 0 333629647 591368192 131007 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 144377 131007 193 433 0 143941 0
[pid=17952/tid=17954] ppid=17950 vsize=577508 CPUtime=1598.06 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 128588 0 0 0 159721 85 0 0 20 0 2 0 333629648 591368192 131007 33554432000 4194304 5964350 140736480749568 140540007499768 4220672 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1601.9
Current children cumulated vsize (KiB) 577508

[startup+1662.3 s]
/proc/loadavg: 2.00 1.99 1.83 3/179 17967
/proc/meminfo: memFree=31017768/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=577760 CPUtime=1661.89 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 129552 0 0 0 166128 61 0 0 20 0 2 0 333629647 591626240 131062 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 144440 131062 193 433 0 144004 0
[pid=17952/tid=17954] ppid=17950 vsize=577760 CPUtime=1657.91 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 128648 0 0 0 165705 86 0 0 20 0 2 0 333629648 591626240 131062 33554432000 4194304 5964350 140736480749568 140540007499816 4400035 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1661.89
Current children cumulated vsize (KiB) 577760

[startup+1722.3 s]
/proc/loadavg: 2.06 2.00 1.84 3/179 17967
/proc/meminfo: memFree=30990952/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=579924 CPUtime=1721.87 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 136168 0 0 0 172121 66 0 0 20 0 2 0 333629647 593842176 137685 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 144981 137685 193 433 0 144545 0
[pid=17952/tid=17954] ppid=17950 vsize=579924 CPUtime=1717.74 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 135264 0 0 0 171682 92 0 0 20 0 2 0 333629648 593842176 137685 33554432000 4194304 5964350 140736480749568 140540007499576 4388689 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1721.87
Current children cumulated vsize (KiB) 579924

[startup+1782.3 s]
/proc/loadavg: 2.02 2.00 1.84 3/179 17967
/proc/meminfo: memFree=30989588/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=581452 CPUtime=1781.86 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 136520 0 0 0 178120 66 0 0 20 0 2 0 333629647 595406848 138032 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 145363 138032 193 433 0 144927 0
[pid=17952/tid=17954] ppid=17950 vsize=581452 CPUtime=1777.59 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 135616 0 0 0 177666 93 0 0 20 0 2 0 333629648 595406848 138032 33554432000 4194304 5964350 140736480749568 140540007499816 4399926 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1781.86
Current children cumulated vsize (KiB) 581452



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1800.5 s]
/proc/loadavg: 2.02 2.00 1.85 2/173 17994
/proc/meminfo: memFree=31205164/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=581692 CPUtime=1800.06 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 136545 0 0 0 179940 66 0 0 20 0 2 0 333629647 595652608 138057 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 145423 138057 193 433 0 144987 0
[pid=17952/tid=17954] ppid=17950 vsize=581692 CPUtime=1795.75 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 135641 0 0 0 179482 93 0 0 20 0 2 0 333629648 595652608 138057 33554432000 4194304 5964350 140736480749568 140540007499816 4379168 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1800.06
Current children cumulated vsize (KiB) 581692

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1800.5 s]
/proc/loadavg: 2.02 2.00 1.85 2/173 17994
/proc/meminfo: memFree=31205164/32873844 swapFree=8656/8656
[pid=17952] ppid=17950 vsize=581692 CPUtime=1800.06 cores=1,3,5,7
/proc/17952/stat : 17952 (wbo1.72) S 17950 17952 17352 0 -1 4202496 136545 0 0 0 179940 66 0 0 20 0 2 0 333629647 595652608 138057 33554432000 4194304 5964350 140736480749568 140736480749088 4546605 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/17952/statm: 145423 138057 193 433 0 144987 0
[pid=17952/tid=17954] ppid=17950 vsize=581692 CPUtime=1795.75 cores=1,3,5,7
/proc/17952/task/17954/stat : 17954 (wbo1.72) R 17950 17952 17352 0 -1 4202560 135641 0 0 0 179482 93 0 0 20 0 2 0 333629648 595652608 138057 33554432000 4194304 5964350 140736480749568 140540007499816 4379168 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1800.06
Current children cumulated vsize (KiB) 581692

Child status: 0

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 17952 and gives
#  childrusage.ru_utime.tv_sec=1794
#  childrusage.ru_utime.tv_usec=841142
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=940856
# CPU time returned by wait4() is 1795.78
# while last known CPU time is 1800.06
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1800.52
CPU time (s): 1800.06
CPU user time (s): 1799.4
CPU system time (s): 0.66
CPU usage (%): 99.9746
Max. virtual memory (cumulated for all children) (KiB): 581692

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1794.84
system time used= 0.940856
maximum resident set size= 552236
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 136547
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 210
involuntary context switches= 1908

runsolver used 3.82242 second user time and 9.95249 second system time

The end

Launcher Data 

Begin job on node102 at 2012-06-03 00:10:11
IDJOB=3728391
IDBENCH=79163
IDSOLVER=2315
FILE ID=node102/3728391-1338675011
RUNJOBID= node102-1338675011-17924
PBS_JOBID= 14636516
Free space on /tmp= 71172 MiB

SOLVER NAME= wbo 1.72
BENCH NAME= PB10/normalized-PB10/SOFT-SMALLINT-LIN/PB06/submitted-PB05/aloul/FPGA_SAT05/normalized-chnl30_40_pb.cnf.cr--soft-0-100-0.wbo
COMMAND LINE= wbo1.72 -file-format=wbo BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3728391-1338675011/watcher-3728391-1338675011 -o /tmp/evaluation-result-3728391-1338675011/solver-3728391-1338675011 -C 1800 -W 1900 -M 15500  wbo1.72 -file-format=wbo HOME/instance-3728391-1338675011.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 68a136cd2642dd27a3fdbf065e2a5463
RANDOM SEED=1713808257

node102.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.178
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.35
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.178
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.178
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.178
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.178
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.178
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5438.54
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.178
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.178
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        31752340 kB
Buffers:          128256 kB
Cached:           232904 kB
SwapCached:         5544 kB
Active:           109524 kB
Inactive:         262272 kB
Active(anon):       7544 kB
Inactive(anon):     5524 kB
Active(file):     101980 kB
Inactive(file):   256748 kB
Unevictable:        8656 kB
Mlocked:            8656 kB
SwapTotal:      67108856 kB
SwapFree:       67097164 kB
Dirty:             10896 kB
Writeback:             0 kB
AnonPages:         14828 kB
Mapped:             9668 kB
Shmem:                 0 kB
Slab:             607632 kB
SReclaimable:      48672 kB
SUnreclaim:       558960 kB
KernelStack:        1384 kB
PageTables:         4348 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     142516 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 71172 MiB
End job on node102 at 2012-06-03 00:40:14