Trace number 3746820

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
npSolver inc-topDown (fixed)? (TO) 1800 1801.02

General information on the benchmark

Name/OPT-BIGINT-LIN/
heinz/normalized-t1722.opb
MD5SUM5a4916943480b697e6c7cc05543abfc1
Bench CategoryOPT-BIGINT-LIN (optimisation, big integers, linear constraints)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark194040
Best CPU time to get the best result obtained on this benchmark1800.15
Has Objective FunctionYES
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables36630
Total number of constraints338
Number of constraints which are clauses0
Number of constraints which are cardinality constraints (but not clauses)338
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint389
Maximum length of a constraint420
Number of terms in the objective function 36630
Biggest coefficient in the objective function 4032
Number of bits for the biggest coefficient in the objective function 12
Sum of the numbers in the objective function 85788512
Number of bits of the sum of numbers in the objective function 27
Biggest number in a constraint 4032
Number of bits of the biggest number in a constraint 12
Biggest sum of numbers in a constraint 85788512
Number of bits of the biggest sum of numbers27
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.27	c run inc solver
0.00/0.27	c [INC SYSTEM] run initial solver
0.00/0.27	c [INC SYSTEM] preprocess with white file
2.92/3.49	c [INC SYSTEM] finished preprocessing in 3 s
2.92/3.49	c [INC SYSTEM] execute solver
2.93/4.11	c [INC SEARCH] formula is SAT
2.93/4.12	c [INC SYSTEM] solved the initial formula
2.93/4.18	c model valid
2.93/4.18	c current solution ok
4.24/4.80	o 276963
4.24/4.80	c [INC SYSTEM] there exists a inc, kill it!
4.24/4.80	c write failed with 2
4.24/4.80	c [INC SEARCH] iteration: 0
4.24/4.80	c [INC BOUNDS] best value:276963 search for 276962
4.24/4.80	c [INC SYSTEM] create new solver instance
4.24/4.80	c [INC] encode full formula
4.24/4.80	c fallback to adder encoding
8.04/8.65	c set iterative enc to: adder
8.04/8.65	c set iterative enc to: none
8.43/9.09	c [INC SEARCH] isOptimum variable: 0
1222.28/1223.14	c formula is SAT
1222.37/1223.23	c [INC SEARCH] copied best model
1222.37/1223.23	c [INC SEARCH] sat variable: 1 optimum/unsat variable: 0
1222.37/1223.30	c model valid
1222.37/1223.30	c current solution ok
1222.37/1223.30	o 276062
1222.37/1223.30	c [INC SEARCH] iteration: 1
1222.37/1223.30	c [INC BOUNDS] best value:276062 search for 276061
1222.37/1223.30	c [INC SYSTEM] there exists a inc, kill it!
1222.37/1223.30	c write failed with 2
1222.47/1223.30	c [INC SYSTEM] create new solver instance
1222.47/1223.30	c [INC] encode full formula
1222.47/1223.30	c fallback to adder encoding
1226.40/1227.26	c set iterative enc to: adder
1226.40/1227.26	c set iterative enc to: none
1226.99/1227.85	c [INC SEARCH] isOptimum variable: 0
1800.00/1801.02	HOME/npSolver-pbo-Itd.sh: line 18: 16243 Terminated              ./npSolver-pbo $file $tmp -rnd_seed=$seed -useQuickBound=0 -useBinSearch=0 -useCP2=1 -useInc=1

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3746820-1339105626/watcher-3746820-1339105626 -o /tmp/evaluation-result-3746820-1339105626/solver-3746820-1339105626 -C 1800 -W 1900 -M 15500 HOME/npSolver-pbo-Itd.sh HOME/instance-3746820-1339105626.opb 1533349139 HOME HOME 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.46 2.81 2.28 2/178 16242
/proc/meminfo: memFree=8670336/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0

[startup+0.057049 s]
/proc/loadavg: 4.46 2.81 2.28 2/178 16242
/proc/meminfo: memFree=8670336/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.100305 s]
/proc/loadavg: 4.46 2.81 2.28 2/178 16242
/proc/meminfo: memFree=8670336/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.300282 s]
/proc/loadavg: 4.46 2.81 2.28 2/178 16242
/proc/meminfo: memFree=8670336/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.70025 s]
/proc/loadavg: 4.46 2.81 2.28 2/178 16242
/proc/meminfo: memFree=8670336/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+1.50045 s]
/proc/loadavg: 4.42 2.83 2.29 2/182 16246
/proc/meminfo: memFree=8506336/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0
[pid=16243] ppid=16242 vsize=26560 CPUtime=0.31 cores=1,3,5,7
/proc/16243/stat : 16243 (npSolver-pbo) S 16242 16242 15254 0 -1 4202496 6229 0 0 0 25 6 0 0 20 0 1 0 376691255 27197440 5943 33554432000 4194304 6004724 140735829080656 140735829075120 5186782 0 65536 4102 0 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/16243/statm: 6640 5943 147 442 0 6194 0
[pid=16245] ppid=16243 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16245/stat : 16245 (cp2_pre.sh) S 16243 16242 15254 0 -1 4202496 606 0 0 0 0 0 0 0 20 0 1 0 376691288 9367552 276 33554432000 4194304 5064252 140737330365136 140737330363840 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 7 0 0 0 0 0
/proc/16245/statm: 2287 276 234 213 0 72 0
[pid=16246] ppid=16245 vsize=88932 CPUtime=1.16 cores=1,3,5,7
/proc/16246/stat : 16246 (coprocessor) R 16245 16242 15254 0 -1 4202496 17531 0 0 0 114 2 0 0 20 0 1 0 376691288 91066368 21509 33554432000 4194304 6586260 140735041801568 140735041797592 4676794 0 0 4096 8404994 0 0 0 17 3 0 0 0 0 0
/proc/16246/statm: 22233 21509 190 584 0 21645 0
Current children cumulated CPU time (s) 1.47
Current children cumulated vsize (KiB) 133788

[startup+3.10036 s]
/proc/loadavg: 4.42 2.83 2.29 2/182 16246
/proc/meminfo: memFree=8435272/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0
[pid=16243] ppid=16242 vsize=26560 CPUtime=0.31 cores=1,3,5,7
/proc/16243/stat : 16243 (npSolver-pbo) S 16242 16242 15254 0 -1 4202496 6229 0 0 0 25 6 0 0 20 0 1 0 376691255 27197440 5943 33554432000 4194304 6004724 140735829080656 140735829075120 5186782 0 65536 4102 0 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/16243/statm: 6640 5943 147 442 0 6194 0
[pid=16245] ppid=16243 vsize=9148 CPUtime=2.61 cores=1,3,5,7
/proc/16245/stat : 16245 (cp2_pre.sh) S 16243 16242 15254 0 -1 4202496 631 17750 0 0 0 0 258 3 20 0 1 0 376691288 9367552 280 33554432000 4194304 5064252 140737330365136 140737330363840 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 7 0 0 0 0 0
/proc/16245/statm: 2287 280 238 213 0 72 0
Current children cumulated CPU time (s) 2.92
Current children cumulated vsize (KiB) 44856

[startup+6.30026 s]
/proc/loadavg: 4.42 2.83 2.29 2/180 16249
/proc/meminfo: memFree=8252308/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0
[pid=16243] ppid=16242 vsize=192972 CPUtime=5.73 cores=1,3,5,7
/proc/16243/stat : 16243 (npSolver-pbo) R 16242 16242 15254 0 -1 4202496 16939 27364 0 0 188 60 318 7 20 0 1 0 376691255 197603328 40706 33554432000 4194304 6004724 140735829080656 140735829073448 4534752 0 0 4096 0 0 0 0 17 1 0 0 0 0 0
/proc/16243/statm: 48243 40706 176 442 0 47797 0
Current children cumulated CPU time (s) 5.73
Current children cumulated vsize (KiB) 202120

[startup+12.7004 s]
/proc/loadavg: 4.51 2.90 2.32 2/181 16251
/proc/meminfo: memFree=7168804/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0
[pid=16243] ppid=16242 vsize=26632 CPUtime=8.53 cores=1,3,5,7
/proc/16243/stat : 16243 (npSolver-pbo) S 16242 16242 15254 0 -1 4202496 18648 27878 0 0 419 108 318 8 20 0 1 0 376691255 27271168 6043 33554432000 4194304 6004724 140735829080656 140735829066504 5190736 0 0 4096 0 18446744071580380219 0 0 17 1 0 0 0 0 0
/proc/16243/statm: 6658 6043 177 442 0 6212 0
[pid=16251] ppid=16243 vsize=939444 CPUtime=3.59 cores=1,3,5,7
/proc/16251/stat : 16251 (glucosInc) R 16243 16242 15254 0 -1 4202496 96639 0 0 0 333 26 0 0 20 0 1 0 376692164 961990656 201612 33554432000 4194304 5654031 140737389490000 140737388439304 4225814 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/16251/statm: 234861 201612 127 357 0 234501 0
Current children cumulated CPU time (s) 12.12
Current children cumulated vsize (KiB) 975224

[startup+25.5004 s]
/proc/loadavg: 4.43 2.93 2.34 2/181 16251
/proc/meminfo: memFree=6921332/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0
[pid=16243] ppid=16242 vsize=26632 CPUtime=8.53 cores=1,3,5,7
/proc/16243/stat : 16243 (npSolver-pbo) S 16242 16242 15254 0 -1 4202496 18648 27878 0 0 419 108 318 8 20 0 1 0 376691255 27271168 6043 33554432000 4194304 6004724 140735829080656 140735829066504 5190736 0 0 4096 0 18446744071580380219 0 0 17 1 0 0 0 0 0
/proc/16243/statm: 6658 6043 177 442 0 6212 0
[pid=16251] ppid=16243 vsize=747748 CPUtime=16.39 cores=1,3,5,7
/proc/16251/stat : 16251 (glucosInc) R 16243 16242 15254 0 -1 4202496 103897 0 0 0 1611 28 0 0 20 0 1 0 376692164 765693952 160534 33554432000 4194304 5654031 140737389490000 140737389477816 4232912 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/16251/statm: 186937 160534 139 357 0 186577 0
Current children cumulated CPU time (s) 24.92
Current children cumulated vsize (KiB) 783528

[startup+51.1004 s]
/proc/loadavg: 4.44 3.05 2.39 2/181 16251
/proc/meminfo: memFree=5078972/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0
[pid=16243] ppid=16242 vsize=26632 CPUtime=8.53 cores=1,3,5,7
/proc/16243/stat : 16243 (npSolver-pbo) S 16242 16242 15254 0 -1 4202496 18648 27878 0 0 419 108 318 8 20 0 1 0 376691255 27271168 6043 33554432000 4194304 6004724 140735829080656 140735829066504 5190736 0 0 4096 0 18446744071580380219 0 0 17 1 0 0 0 0 0
/proc/16243/statm: 6658 6043 177 442 0 6212 0
[pid=16251] ppid=16243 vsize=747880 CPUtime=41.98 cores=1,3,5,7
/proc/16251/stat : 16251 (glucosInc) R 16243 16242 15254 0 -1 4202496 103951 0 0 0 4170 28 0 0 20 0 1 0 376692164 765829120 160588 33554432000 4194304 5654031 140737389490000 140737389487784 4232453 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/16251/statm: 186970 160588 139 357 0 186610 0
Current children cumulated CPU time (s) 50.51
Current children cumulated vsize (KiB) 783660

[startup+102.3 s]
/proc/loadavg: 4.77 3.38 2.54 3/181 16251
/proc/meminfo: memFree=4912904/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0
[pid=16243] ppid=16242 vsize=26632 CPUtime=8.53 cores=1,3,5,7
/proc/16243/stat : 16243 (npSolver-pbo) S 16242 16242 15254 0 -1 4202496 18648 27878 0 0 419 108 318 8 20 0 1 0 376691255 27271168 6043 33554432000 4194304 6004724 140735829080656 140735829066504 5190736 0 0 4096 0 18446744071580380219 0 0 17 1 0 0 0 0 0
/proc/16243/statm: 6658 6043 177 442 0 6212 0
[pid=16251] ppid=16243 vsize=753676 CPUtime=93.18 cores=1,3,5,7
/proc/16251/stat : 16251 (glucosInc) R 16243 16242 15254 0 -1 4202496 105627 0 0 0 9289 29 0 0 20 0 1 0 376692164 771764224 165323 33554432000 4194304 5654031 140737389490000 140737389487784 4232775 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/16251/statm: 188419 165323 141 357 0 188059 0
Current children cumulated CPU time (s) 101.71
Current children cumulated vsize (KiB) 789456

[startup+162.3 s]
/proc/loadavg: 4.13 3.49 2.63 3/181 16291
/proc/meminfo: memFree=28195644/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0
[pid=16243] ppid=16242 vsize=26632 CPUtime=8.53 cores=1,3,5,7
/proc/16243/stat : 16243 (npSolver-pbo) S 16242 16242 15254 0 -1 4202496 18648 27878 0 0 419 108 318 8 20 0 1 0 376691255 27271168 6043 33554432000 4194304 6004724 140735829080656 140735829066504 5190736 0 0 4096 0 18446744071580380219 0 0 17 1 0 0 0 0 0
/proc/16243/statm: 6658 6043 177 442 0 6212 0
[pid=16251] ppid=16243 vsize=761628 CPUtime=153.17 cores=1,3,5,7
/proc/16251/stat : 16251 (glucosInc) R 16243 16242 15254 0 -1 4202496 107221 0 0 0 15287 30 0 0 20 0 1 0 376692164 779907072 170445 33554432000 4194304 5654031 140737389490000 140737389477816 4232912 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/16251/statm: 190407 170445 141 357 0 190047 0
Current children cumulated CPU time (s) 161.7
Current children cumulated vsize (KiB) 797408

[startup+222.3 s]
/proc/loadavg: 2.78 3.22 2.59 3/181 16293
/proc/meminfo: memFree=23469236/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0
[pid=16243] ppid=16242 vsize=26632 CPUtime=8.53 cores=1,3,5,7
/proc/16243/stat : 16243 (npSolver-pbo) S 16242 16242 15254 0 -1 4202496 18648 27878 0 0 419 108 318 8 20 0 1 0 376691255 27271168 6043 33554432000 4194304 6004724 140735829080656 140735829066504 5190736 0 0 4096 0 18446744071580380219 0 0 17 1 0 0 0 0 0
/proc/16243/statm: 6658 6043 177 442 0 6212 0
[pid=16251] ppid=16243 vsize=763952 CPUtime=213.14 cores=1,3,5,7
/proc/16251/stat : 16251 (glucosInc) R 16243 16242 15254 0 -1 4202496 107932 0 0 0 21283 31 0 0 20 0 1 0 376692164 782286848 175223 33554432000 4194304 5654031 140737389490000 140737389477816 4232800 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/16251/statm: 190988 175223 141 357 0 190628 0
Current children cumulated CPU time (s) 221.67
Current children cumulated vsize (KiB) 799732

[startup+282.307 s]

################
# More data... #
################

/proc/loadavg: 2.01 2.13 2.42 3/181 16424
/proc/meminfo: memFree=29100264/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0
[pid=16243] ppid=16242 vsize=65636 CPUtime=1227.05 cores=1,3,5,7
/proc/16243/stat : 16243 (npSolver-pbo) S 16242 16242 15254 0 -1 4202496 28632 146210 0 0 830 168 121640 67 20 0 1 0 376691255 67211264 15794 33554432000 4194304 6004724 140735829080656 140735829066504 5190736 0 0 4096 0 18446744071580380219 0 0 17 1 0 0 0 0 0
/proc/16243/statm: 16409 15794 177 442 0 15963 0
[pid=16381] ppid=16243 vsize=758172 CPUtime=134.34 cores=1,3,5,7
/proc/16381/stat : 16381 (glucosInc) R 16243 16242 15254 0 -1 4202496 105938 0 0 0 13397 37 0 0 20 0 1 0 376814041 776368128 171097 33554432000 4194304 5654031 140735556953840 140735556941656 4232622 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/16381/statm: 189543 171097 141 357 0 189183 0
Current children cumulated CPU time (s) 1361.39
Current children cumulated vsize (KiB) 832956

[startup+1422.3 s]
/proc/loadavg: 2.00 2.10 2.39 3/182 16427
/proc/meminfo: memFree=29070072/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0
[pid=16243] ppid=16242 vsize=65636 CPUtime=1227.05 cores=1,3,5,7
/proc/16243/stat : 16243 (npSolver-pbo) S 16242 16242 15254 0 -1 4202496 28632 146210 0 0 830 168 121640 67 20 0 1 0 376691255 67211264 15794 33554432000 4194304 6004724 140735829080656 140735829066504 5190736 0 0 4096 0 18446744071580380219 0 0 17 1 0 0 0 0 0
/proc/16243/statm: 16409 15794 177 442 0 15963 0
[pid=16381] ppid=16243 vsize=759044 CPUtime=194.32 cores=1,3,5,7
/proc/16381/stat : 16381 (glucosInc) R 16243 16242 15254 0 -1 4202496 106111 0 0 0 19395 37 0 0 20 0 1 0 376814041 777261056 172454 33554432000 4194304 5654031 140735556953840 140735556941656 4232800 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/16381/statm: 189761 172454 141 357 0 189401 0
Current children cumulated CPU time (s) 1421.37
Current children cumulated vsize (KiB) 833828

[startup+1482.3 s]
/proc/loadavg: 2.00 2.08 2.37 3/181 16427
/proc/meminfo: memFree=29068816/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0
[pid=16243] ppid=16242 vsize=65636 CPUtime=1227.05 cores=1,3,5,7
/proc/16243/stat : 16243 (npSolver-pbo) S 16242 16242 15254 0 -1 4202496 28632 146210 0 0 830 168 121640 67 20 0 1 0 376691255 67211264 15794 33554432000 4194304 6004724 140735829080656 140735829066504 5190736 0 0 4096 0 18446744071580380219 0 0 17 1 0 0 0 0 0
/proc/16243/statm: 16409 15794 177 442 0 15963 0
[pid=16381] ppid=16243 vsize=760480 CPUtime=254.31 cores=1,3,5,7
/proc/16381/stat : 16381 (glucosInc) R 16243 16242 15254 0 -1 4202496 106479 0 0 0 25394 37 0 0 20 0 1 0 376814041 778731520 174866 33554432000 4194304 5654031 140735556953840 140735556941656 4232421 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/16381/statm: 190120 174866 141 357 0 189760 0
Current children cumulated CPU time (s) 1481.36
Current children cumulated vsize (KiB) 835264

[startup+1542.3 s]
/proc/loadavg: 2.00 2.06 2.34 3/181 16428
/proc/meminfo: memFree=29060784/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0
[pid=16243] ppid=16242 vsize=65636 CPUtime=1227.05 cores=1,3,5,7
/proc/16243/stat : 16243 (npSolver-pbo) S 16242 16242 15254 0 -1 4202496 28632 146210 0 0 830 168 121640 67 20 0 1 0 376691255 67211264 15794 33554432000 4194304 6004724 140735829080656 140735829066504 5190736 0 0 4096 0 18446744071580380219 0 0 17 1 0 0 0 0 0
/proc/16243/statm: 16409 15794 177 442 0 15963 0
[pid=16381] ppid=16243 vsize=764120 CPUtime=314.31 cores=1,3,5,7
/proc/16381/stat : 16381 (glucosInc) R 16243 16242 15254 0 -1 4202496 107809 0 0 0 31390 41 0 0 20 0 1 0 376814041 782458880 166134 33554432000 4194304 5654031 140735556953840 140735556941656 4232707 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/16381/statm: 191030 166134 141 357 0 190670 0
Current children cumulated CPU time (s) 1541.36
Current children cumulated vsize (KiB) 838904

[startup+1602.3 s]
/proc/loadavg: 2.06 2.06 2.32 3/181 16428
/proc/meminfo: memFree=29100716/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0
[pid=16243] ppid=16242 vsize=65636 CPUtime=1227.05 cores=1,3,5,7
/proc/16243/stat : 16243 (npSolver-pbo) S 16242 16242 15254 0 -1 4202496 28632 146210 0 0 830 168 121640 67 20 0 1 0 376691255 67211264 15794 33554432000 4194304 6004724 140735829080656 140735829066504 5190736 0 0 4096 0 18446744071580380219 0 0 17 1 0 0 0 0 0
/proc/16243/statm: 16409 15794 177 442 0 15963 0
[pid=16381] ppid=16243 vsize=764808 CPUtime=374.28 cores=1,3,5,7
/proc/16381/stat : 16381 (glucosInc) R 16243 16242 15254 0 -1 4202496 108113 0 0 0 37387 41 0 0 20 0 1 0 376814041 783163392 169997 33554432000 4194304 5654031 140735556953840 140735556941656 4232939 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/16381/statm: 191202 169997 141 357 0 190842 0
Current children cumulated CPU time (s) 1601.33
Current children cumulated vsize (KiB) 839592

[startup+1662.3 s]
/proc/loadavg: 2.02 2.05 2.30 3/181 16428
/proc/meminfo: memFree=29065776/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0
[pid=16243] ppid=16242 vsize=65636 CPUtime=1227.05 cores=1,3,5,7
/proc/16243/stat : 16243 (npSolver-pbo) S 16242 16242 15254 0 -1 4202496 28632 146210 0 0 830 168 121640 67 20 0 1 0 376691255 67211264 15794 33554432000 4194304 6004724 140735829080656 140735829066504 5190736 0 0 4096 0 18446744071580380219 0 0 17 1 0 0 0 0 0
/proc/16243/statm: 16409 15794 177 442 0 15963 0
[pid=16381] ppid=16243 vsize=767056 CPUtime=434.27 cores=1,3,5,7
/proc/16381/stat : 16381 (glucosInc) R 16243 16242 15254 0 -1 4202496 108684 0 0 0 43385 42 0 0 20 0 1 0 376814041 785465344 174308 33554432000 4194304 5654031 140735556953840 140735556941656 4232821 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/16381/statm: 191764 174308 141 357 0 191404 0
Current children cumulated CPU time (s) 1661.32
Current children cumulated vsize (KiB) 841840

[startup+1722.3 s]
/proc/loadavg: 2.00 2.04 2.28 3/181 16428
/proc/meminfo: memFree=29040164/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0
[pid=16243] ppid=16242 vsize=65636 CPUtime=1227.05 cores=1,3,5,7
/proc/16243/stat : 16243 (npSolver-pbo) S 16242 16242 15254 0 -1 4202496 28632 146210 0 0 830 168 121640 67 20 0 1 0 376691255 67211264 15794 33554432000 4194304 6004724 140735829080656 140735829066504 5190736 0 0 4096 0 18446744071580380219 0 0 17 1 0 0 0 0 0
/proc/16243/statm: 16409 15794 177 442 0 15963 0
[pid=16381] ppid=16243 vsize=769276 CPUtime=494.25 cores=1,3,5,7
/proc/16381/stat : 16381 (glucosInc) R 16243 16242 15254 0 -1 4202496 108901 0 0 0 49383 42 0 0 20 0 1 0 376814041 787738624 176129 33554432000 4194304 5654031 140735556953840 140735556941656 4232775 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/16381/statm: 192319 176129 141 357 0 191959 0
Current children cumulated CPU time (s) 1721.3
Current children cumulated vsize (KiB) 844060

[startup+1782.3 s]
/proc/loadavg: 2.00 2.03 2.26 3/181 16428
/proc/meminfo: memFree=29019984/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0
[pid=16243] ppid=16242 vsize=65636 CPUtime=1227.05 cores=1,3,5,7
/proc/16243/stat : 16243 (npSolver-pbo) S 16242 16242 15254 0 -1 4202496 28632 146210 0 0 830 168 121640 67 20 0 1 0 376691255 67211264 15794 33554432000 4194304 6004724 140735829080656 140735829066504 5190736 0 0 4096 0 18446744071580380219 0 0 17 1 0 0 0 0 0
/proc/16243/statm: 16409 15794 177 442 0 15963 0
[pid=16381] ppid=16243 vsize=770716 CPUtime=554.25 cores=1,3,5,7
/proc/16381/stat : 16381 (glucosInc) R 16243 16242 15254 0 -1 4202496 109499 0 0 0 55382 43 0 0 20 0 1 0 376814041 789213184 179793 33554432000 4194304 5654031 140735556953840 140735556941656 4208595 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/16381/statm: 192679 179793 141 357 0 192319 0
Current children cumulated CPU time (s) 1781.3
Current children cumulated vsize (KiB) 845500



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1801 s]
/proc/loadavg: 2.00 2.03 2.25 3/181 16428
/proc/meminfo: memFree=28981196/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0
[pid=16243] ppid=16242 vsize=65636 CPUtime=1227.05 cores=1,3,5,7
/proc/16243/stat : 16243 (npSolver-pbo) S 16242 16242 15254 0 -1 4202496 28632 146210 0 0 830 168 121640 67 20 0 1 0 376691255 67211264 15794 33554432000 4194304 6004724 140735829080656 140735829066504 5190736 0 0 4096 0 18446744071580380219 0 0 17 1 0 0 0 0 0
/proc/16243/statm: 16409 15794 177 442 0 15963 0
[pid=16381] ppid=16243 vsize=770716 CPUtime=572.95 cores=1,3,5,7
/proc/16381/stat : 16381 (glucosInc) R 16243 16242 15254 0 -1 4202496 109500 0 0 0 57252 43 0 0 20 0 1 0 376814041 789213184 180305 33554432000 4194304 5654031 140735556953840 140735556941656 4208595 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/16381/statm: 192679 180305 141 357 0 192319 0
Current children cumulated CPU time (s) 1800
Current children cumulated vsize (KiB) 845500

Sending SIGTERM to process tree (bottom up)

Solver just ended. Dumping a history of the last processes samples

[startup+1801 s]
/proc/loadavg: 2.00 2.03 2.25 3/181 16428
/proc/meminfo: memFree=28981196/32873844 swapFree=7116/7116
[pid=16242] ppid=16240 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/16242/stat : 16242 (npSolver-pbo-It) S 16240 16242 15254 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 376691255 9367552 274 33554432000 4194304 5064252 140733733620544 140733733619456 247064061518 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/16242/statm: 2287 274 233 213 0 72 0
[pid=16243] ppid=16242 vsize=65636 CPUtime=1227.05 cores=1,3,5,7
/proc/16243/stat : 16243 (npSolver-pbo) S 16242 16242 15254 0 -1 4202496 28632 146210 0 0 830 168 121640 67 20 0 1 0 376691255 67211264 15794 33554432000 4194304 6004724 140735829080656 140735829066504 5190736 0 0 4096 0 18446744071580380219 0 0 17 1 0 0 0 0 0
/proc/16243/statm: 16409 15794 177 442 0 15963 0
[pid=16381] ppid=16243 vsize=770716 CPUtime=572.95 cores=1,3,5,7
/proc/16381/stat : 16381 (glucosInc) R 16243 16242 15254 0 -1 4202496 109500 0 0 0 57252 43 0 0 20 0 1 0 376814041 789213184 180305 33554432000 4194304 5654031 140735556953840 140735556941656 4208595 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/16381/statm: 192679 180305 141 357 0 192319 0
Current children cumulated CPU time (s) 1800
Current children cumulated vsize (KiB) 845500

Child status: 0

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 16242 and gives
#  childrusage.ru_utime.tv_sec=1224
#  childrusage.ru_utime.tv_usec=705816
#  childrusage.ru_stime.tv_sec=2
#  childrusage.ru_stime.tv_usec=366640
# CPU time returned by wait4() is 1227.07
# while last known CPU time is 1800
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1801.02
CPU time (s): 1800
CPU user time (s): 1797.22
CPU system time (s): 2.78
CPU usage (%): 99.9433
Max. virtual memory (cumulated for all children) (KiB): 1171956

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1224.71
system time used= 2.36664
maximum resident set size= 935480
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 175216
page faults= 0
swaps= 0
block input operations= 0
block output operations= 1166552
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3190
involuntary context switches= 1302

runsolver used 3.42148 second user time and 8.66968 second system time

The end

Launcher Data 

Begin job on node103 at 2012-06-07 23:47:07
IDJOB=3746820
IDBENCH=89896
IDSOLVER=2332
FILE ID=node103/3746820-1339105626
RUNJOBID= node103-1339098177-15271
PBS_JOBID= 14643750
Free space on /tmp= 65348 MiB

SOLVER NAME= npSolver inc-topDown
BENCH NAME= PB11/normalized-PB11/OPT-BIGINT-LIN/heinz/normalized-t1722.opb
COMMAND LINE= DIR/npSolver-pbo-Itd.sh BENCHNAME RANDOMSEED TMPDIR DIR
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3746820-1339105626/watcher-3746820-1339105626 -o /tmp/evaluation-result-3746820-1339105626/solver-3746820-1339105626 -C 1800 -W 1900 -M 15500  HOME/npSolver-pbo-Itd.sh HOME/instance-3746820-1339105626.opb 1533349139 HOME HOME

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 5a4916943480b697e6c7cc05543abfc1
RANDOM SEED=1533349139

node103.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.189
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.37
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.189
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.189
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.189
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.189
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.189
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5411.36
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.189
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.189
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:         8670716 kB
Buffers:          264392 kB
Cached:         13402820 kB
SwapCached:         5468 kB
Active:          9591708 kB
Inactive:       13196988 kB
Active(anon):    9118048 kB
Inactive(anon):     5816 kB
Active(file):     473660 kB
Inactive(file): 13191172 kB
Unevictable:        7116 kB
Mlocked:            7116 kB
SwapTotal:      67108856 kB
SwapFree:       67096900 kB
Dirty:           5787180 kB
Writeback:             0 kB
AnonPages:       9124536 kB
Mapped:            10024 kB
Shmem:                 0 kB
Slab:            1254692 kB
SReclaimable:     690816 kB
SUnreclaim:       563876 kB
KernelStack:        1448 kB
PageTables:        21908 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:    9648856 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:   8302592 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70936 MiB
End job on node103 at 2012-06-08 00:17:08