Trace number 4103775

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
NaPS 1.02? (TO) 1800.02 1802.3

General information on the benchmark

Namenormalized-PB06/OPT-BIGINT/mps-v2-20-10/ftp.netlib.org/
lp/data/normalized-mps-v2-20-10-agg.opb
MD5SUM4f19aa044e880280faec0077a2509ba8
Bench CategoryOPT-BIGINT (optimisation, big integers)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Has Objective FunctionYES
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables4890
Total number of constraints363
Number of constraints which are clauses0
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints363
Minimum length of a constraint30
Maximum length of a constraint540
Number of terms in the objective function 3930
Biggest coefficient in the objective function 5373004087296
Number of bits for the biggest coefficient in the objective function 43
Sum of the numbers in the objective function 556141355997381
Number of bits of the sum of numbers in the objective function 49
Biggest number in a constraint 22763326668800000
Number of bits of the biggest number in a constraint 55
Biggest sum of numbers in a constraint 46271983665442689
Number of bits of the biggest sum of numbers56
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c NaPS 1.02b.

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2013 Olivier ROUSSEL

This is runsolver version 3.3.7 (svn: 2553)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4103775-1466266394/watcher-4103775-1466266394 -o /tmp/evaluation-result-4103775-1466266394/solver-4103775-1466266394 -C 1800 -W 1900 -M 15500 --bin-var /tmp/runsolver-exch-node136-1466264583-28002 --watchdog 1960 naps-1.02 HOME/instance-4103775-1466266394.opb 

pid=28279
running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.26 2.30 2.27 2/176 28282
/proc/meminfo: memFree=30152652/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=7296 CPUtime=0 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 475 0 0 0 0 0 0 0 20 0 1 0 28684958 7471104 349 33554432000 4194304 5796924 140734928512672 140735330471160 5095032 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 1824 359 114 392 0 1423 0

[startup+0.100114 s]
/proc/loadavg: 2.26 2.30 2.27 2/176 28282
/proc/meminfo: memFree=30152652/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=9544 CPUtime=0.09 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 1017 0 0 0 9 0 0 0 20 0 1 0 28684958 9773056 891 33554432000 4194304 5796924 140734928512672 140735330471160 5183584 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 2386 891 128 392 0 1985 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 9544

[startup+0.200286 s]
/proc/loadavg: 2.26 2.30 2.27 2/176 28282
/proc/meminfo: memFree=30152652/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=11668 CPUtime=0.19 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 1532 0 0 0 19 0 0 0 20 0 1 0 28684958 11948032 1406 33554432000 4194304 5796924 140734928512672 140734928511832 5087778 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 2917 1406 128 392 0 2516 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 11668

[startup+0.300223 s]
/proc/loadavg: 2.26 2.30 2.27 2/176 28282
/proc/meminfo: memFree=30152652/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=14564 CPUtime=0.29 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 2280 0 0 0 29 0 0 0 20 0 1 0 28684958 14913536 2115 33554432000 4194304 5796924 140734928512672 140734928511832 5095032 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 3641 2115 139 392 0 3240 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 14564

[startup+0.700231 s]
/proc/loadavg: 2.26 2.30 2.27 2/176 28282
/proc/meminfo: memFree=30152652/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=38468 CPUtime=0.69 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 8509 0 0 0 68 1 0 0 20 0 1 0 28684958 39391232 8054 33554432000 4194304 5796924 140734928512672 140734928511832 5095032 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 9617 8054 139 392 0 9216 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 38468

[startup+1.50018 s]
/proc/loadavg: 2.26 2.30 2.27 3/177 28283
/proc/meminfo: memFree=30062348/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=43928 CPUtime=1.49 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 9887 0 0 0 148 1 0 0 20 0 1 0 28684958 44982272 9432 33554432000 4194304 5796924 140734928512672 140734928509656 5087650 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 10982 9432 144 392 0 10581 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 43928

[startup+3.10018 s]
/proc/loadavg: 2.26 2.30 2.27 3/177 28283
/proc/meminfo: memFree=30010516/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=45024 CPUtime=3.09 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 10105 0 0 0 307 2 0 0 20 0 1 0 28684958 46104576 9650 33554432000 4194304 5796924 140734928512672 140734928507160 5087778 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 11256 9650 144 392 0 10855 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 45024

[startup+6.30018 s]
/proc/loadavg: 2.24 2.30 2.27 3/177 28283
/proc/meminfo: memFree=29860744/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=45024 CPUtime=6.29 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 10118 0 0 0 627 2 0 0 20 0 1 0 28684958 46104576 9663 33554432000 4194304 5796924 140734928512672 140734928507240 4865437 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 11256 9663 144 392 0 10855 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 45024

[startup+12.7002 s]
/proc/loadavg: 2.22 2.29 2.27 3/177 28283
/proc/meminfo: memFree=29572540/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=45684 CPUtime=12.69 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 10286 0 0 0 1267 2 0 0 20 0 1 0 28684958 46780416 9831 33554432000 4194304 5796924 140734928512672 140734928507160 4865437 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 11421 9831 144 392 0 11020 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 45684

[startup+25.5002 s]
/proc/loadavg: 2.25 2.29 2.27 3/177 28283
/proc/meminfo: memFree=29021252/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=43636 CPUtime=25.49 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 10293 0 0 0 2547 2 0 0 20 0 1 0 28684958 44683264 9402 33554432000 4194304 5796924 140734928512672 140734928508888 5092526 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 10909 9402 151 392 0 10508 0
Current children cumulated CPU time (s) 25.49
Current children cumulated vsize (KiB) 43636

[startup+51.1003 s]
/proc/loadavg: 2.23 2.28 2.26 3/177 28283
/proc/meminfo: memFree=27937368/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=48452 CPUtime=51.08 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 11787 0 0 0 5106 2 0 0 20 0 1 0 28684958 49614848 10545 33554432000 4194304 5796924 140734928512672 140734928507304 5087558 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 12113 10545 151 392 0 11712 0
Current children cumulated CPU time (s) 51.08
Current children cumulated vsize (KiB) 48452

[startup+102.3 s]
/proc/loadavg: 2.29 2.30 2.27 3/177 28284
/proc/meminfo: memFree=25739776/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=59888 CPUtime=102.29 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 15490 0 0 0 10226 3 0 0 20 0 1 0 28684958 61325312 13372 33554432000 4194304 5796924 140734928512672 140734928507160 4810733 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 14972 13372 151 392 0 14571 0
Current children cumulated CPU time (s) 102.29
Current children cumulated vsize (KiB) 59888

[startup+162.3 s]
/proc/loadavg: 2.37 2.32 2.28 3/177 28285
/proc/meminfo: memFree=23303912/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=63552 CPUtime=162.27 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 17266 0 0 0 16224 3 0 0 20 0 1 0 28684958 65077248 14211 33554432000 4194304 5796924 140734928512672 140734928507304 4809286 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 15888 14211 151 392 0 15487 0
Current children cumulated CPU time (s) 162.27
Current children cumulated vsize (KiB) 63552

[startup+222.307 s]
/proc/loadavg: 2.19 2.27 2.26 3/177 28286
/proc/meminfo: memFree=20879212/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=58864 CPUtime=222.26 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 17271 0 0 0 22223 3 0 0 20 0 1 0 28684958 60276736 13153 33554432000 4194304 5796924 140734928512672 140734928508888 5096620 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 14716 13153 151 392 0 14315 0
Current children cumulated CPU time (s) 222.26
Current children cumulated vsize (KiB) 58864

[startup+282.3 s]
/proc/loadavg: 2.15 2.25 2.26 3/177 28288
/proc/meminfo: memFree=18527792/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=67592 CPUtime=282.25 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 19300 0 0 0 28222 3 0 0 20 0 1 0 28684958 69214208 15182 33554432000 4194304 5796924 140734928512672 140734928507240 4861426 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 16898 15182 151 392 0 16497 0
Current children cumulated CPU time (s) 282.25
Current children cumulated vsize (KiB) 67592

[startup+342.3 s]
/proc/loadavg: 2.20 2.25 2.25 3/177 28289
/proc/meminfo: memFree=16202236/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=69932 CPUtime=342.24 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 19876 0 0 0 34221 3 0 0 20 0 1 0 28684958 71610368 15758 33554432000 4194304 5796924 140734928512672 140734928507160 4296026 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 17483 15758 151 392 0 17082 0
Current children cumulated CPU time (s) 342.24
Current children cumulated vsize (KiB) 69932

[startup+402.3 s]
/proc/loadavg: 2.21 2.23 2.25 3/177 28336
/proc/meminfo: memFree=29843952/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=69932 CPUtime=402.23 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 19888 0 0 0 40220 3 0 0 20 0 1 0 28684958 71610368 15770 33554432000 4194304 5796924 140734928512672 140734928508888 5092518 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 17483 15770 151 392 0 17082 0
Current children cumulated CPU time (s) 402.23
Current children cumulated vsize (KiB) 69932

[startup+462.3 s]
/proc/loadavg: 2.25 2.24 2.25 3/178 28340
/proc/meminfo: memFree=27293560/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=75296 CPUtime=462.22 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 21255 0 0 0 46219 3 0 0 20 0 1 0 28684958 77103104 16989 33554432000 4194304 5796924 140734928512672 140734928509656 5100101 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 18824 16989 151 392 0 18423 0
Current children cumulated CPU time (s) 462.22
Current children cumulated vsize (KiB) 75296

[startup+522.3 s]
/proc/loadavg: 2.21 2.23 2.24 3/177 28341
/proc/meminfo: memFree=24888708/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=75296 CPUtime=522.21 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 21255 0 0 0 52218 3 0 0 20 0 1 0 28684958 77103104 16989 33554432000 4194304 5796924 140734928512672 140734928509656 4865443 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 18824 16989 151 392 0 18423 0
Current children cumulated CPU time (s) 522.21
Current children cumulated vsize (KiB) 75296

[startup+582.3 s]
/proc/loadavg: 2.27 2.24 2.25 3/177 28343
/proc/meminfo: memFree=22490912/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=78572 CPUtime=582.21 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 22183 0 0 0 58217 4 0 0 20 0 1 0 28684958 80457728 17917 33554432000 4194304 5796924 140734928512672 140734928507160 4809393 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 19643 17917 151 392 0 19242 0
Current children cumulated CPU time (s) 582.21
Current children cumulated vsize (KiB) 78572


################
# More data... #
################

[pid=28282] ppid=28279 vsize=125672 CPUtime=1122.1 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 32507 0 0 0 112205 5 0 0 20 0 1 0 28684958 128688128 29759 33554432000 4194304 5796924 140734928512672 140734928507160 5092604 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 31418 29759 151 392 0 31017 0
Current children cumulated CPU time (s) 1122.1
Current children cumulated vsize (KiB) 125672

[startup+1182.3 s]
/proc/loadavg: 2.17 2.20 2.23 3/175 28448
/proc/meminfo: memFree=30136124/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=125672 CPUtime=1182.09 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 32508 0 0 0 118204 5 0 0 20 0 1 0 28684958 128688128 29760 33554432000 4194304 5796924 140734928512672 140734928507160 5092643 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 31418 29760 151 392 0 31017 0
Current children cumulated CPU time (s) 1182.09
Current children cumulated vsize (KiB) 125672

[startup+1242.3 s]
/proc/loadavg: 2.27 2.22 2.23 3/177 28451
/proc/meminfo: memFree=30114204/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=144096 CPUtime=1242.09 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 37381 0 0 0 124203 6 0 0 20 0 1 0 28684958 147554304 34633 33554432000 4194304 5796924 140734928512672 140734928507240 4865437 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 36024 34633 151 392 0 35623 0
Current children cumulated CPU time (s) 1242.09
Current children cumulated vsize (KiB) 144096

[startup+1302.3 s]
/proc/loadavg: 2.20 2.21 2.22 3/177 28452
/proc/meminfo: memFree=29975788/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=144096 CPUtime=1302.08 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 37382 0 0 0 130202 6 0 0 20 0 1 0 28684958 147554304 34634 33554432000 4194304 5796924 140734928512672 140734928508888 4313392 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 36024 34634 151 392 0 35623 0
Current children cumulated CPU time (s) 1302.08
Current children cumulated vsize (KiB) 144096

[startup+1362.3 s]
/proc/loadavg: 2.15 2.19 2.22 3/177 28453
/proc/meminfo: memFree=29851768/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=144288 CPUtime=1362.07 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 37449 0 0 0 136201 6 0 0 20 0 1 0 28684958 147750912 34701 33554432000 4194304 5796924 140734928512672 140734928509656 4809992 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 36072 34701 151 392 0 35671 0
Current children cumulated CPU time (s) 1362.07
Current children cumulated vsize (KiB) 144288

[startup+1422.3 s]
/proc/loadavg: 2.23 2.20 2.22 3/177 28454
/proc/meminfo: memFree=29830376/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=145312 CPUtime=1422.07 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 37619 0 0 0 142201 6 0 0 20 0 1 0 28684958 148799488 34871 33554432000 4194304 5796924 140734928512672 140734928509656 5092651 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 36328 34871 151 392 0 35927 0
Current children cumulated CPU time (s) 1422.07
Current children cumulated vsize (KiB) 145312

[startup+1482.3 s]
/proc/loadavg: 2.14 2.18 2.20 3/177 28455
/proc/meminfo: memFree=29834624/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=145312 CPUtime=1482.05 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 37619 0 0 0 148199 6 0 0 20 0 1 0 28684958 148799488 34871 33554432000 4194304 5796924 140734928512672 140734928509656 4865437 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 36328 34871 151 392 0 35927 0
Current children cumulated CPU time (s) 1482.05
Current children cumulated vsize (KiB) 145312

[startup+1542.31 s]
/proc/loadavg: 2.13 2.17 2.20 3/177 28457
/proc/meminfo: memFree=29825816/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=146628 CPUtime=1542.05 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 38137 0 0 0 154199 6 0 0 20 0 1 0 28684958 150147072 35389 33554432000 4194304 5796924 140734928512672 140734928507160 5096709 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 36657 35389 151 392 0 36256 0
Current children cumulated CPU time (s) 1542.05
Current children cumulated vsize (KiB) 146628

[startup+1602.3 s]
/proc/loadavg: 2.16 2.17 2.19 3/177 28458
/proc/meminfo: memFree=29820168/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=148676 CPUtime=1602.03 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 38396 0 0 0 160197 6 0 0 20 0 1 0 28684958 152244224 35648 33554432000 4194304 5796924 140734928512672 140734928507240 5087606 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 37169 35648 151 392 0 36768 0
Current children cumulated CPU time (s) 1602.03
Current children cumulated vsize (KiB) 148676

[startup+1662.3 s]
/proc/loadavg: 2.23 2.20 2.19 3/178 28462
/proc/meminfo: memFree=29796188/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=148676 CPUtime=1662.02 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 38505 0 0 0 166196 6 0 0 20 0 1 0 28684958 152244224 35757 33554432000 4194304 5796924 140734928512672 140734928509656 5092558 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 37169 35757 151 392 0 36768 0
Current children cumulated CPU time (s) 1662.02
Current children cumulated vsize (KiB) 148676

[startup+1722.3 s]
/proc/loadavg: 2.31 2.22 2.20 3/177 28463
/proc/meminfo: memFree=29795200/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=148676 CPUtime=1722.01 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 38505 0 0 0 172195 6 0 0 20 0 1 0 28684958 152244224 35757 33554432000 4194304 5796924 140734928512672 140734928509656 5092467 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 37169 35757 151 392 0 36768 0
Current children cumulated CPU time (s) 1722.01
Current children cumulated vsize (KiB) 148676

[startup+1782.3 s]
/proc/loadavg: 2.31 2.24 2.20 3/177 28464
/proc/meminfo: memFree=29798852/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=148676 CPUtime=1782 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 38505 0 0 0 178194 6 0 0 20 0 1 0 28684958 152244224 35757 33554432000 4194304 5796924 140734928512672 140734928509656 5092526 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 37169 35757 151 392 0 36768 0
Current children cumulated CPU time (s) 1782
Current children cumulated vsize (KiB) 148676



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1800.3 s]
/proc/loadavg: 2.22 2.22 2.20 3/177 28465
/proc/meminfo: memFree=29790908/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=148676 CPUtime=1800 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) R 28279 28282 27985 0 -1 4202496 38537 0 0 0 179994 6 0 0 20 0 1 0 28684958 152244224 35789 33554432000 4194304 5796924 140734928512672 140734928509656 4199792 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 37169 35789 151 392 0 36768 0
Current children cumulated CPU time (s) 1800
Current children cumulated vsize (KiB) 148676

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -28282
??? exit watcher main loop
??? sigkill at end of watcher main loop
??? kill 28282 sig 9
??? end of watcher thread

Solver just ended.
Dumping a history of the last processes samples

[startup+1801.6 s]
/proc/loadavg: 2.22 2.22 2.20 2/177 28465
/proc/meminfo: memFree=29790912/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=148676 CPUtime=1800 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) S 28279 28282 27985 0 -1 4194304 38538 0 0 0 179994 6 0 0 20 0 1 0 28684958 152244224 35790 33554432000 4194304 5796924 140734928512672 140734928506056 5190702 0 16384 4096 17954 18446744071579510697 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 37169 35790 152 392 0 36768 0
Current children cumulated CPU time (s) 1800
Current children cumulated vsize (KiB) 148676

[startup+1802 s]
/proc/loadavg: 2.22 2.22 2.20 2/177 28465
/proc/meminfo: memFree=29790912/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=148676 CPUtime=1800 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) S 28279 28282 27985 0 -1 4194304 38538 0 0 0 179994 6 0 0 20 0 1 0 28684958 152244224 35790 33554432000 4194304 5796924 140734928512672 140734928506056 5190702 0 16384 4096 17954 18446744071579510697 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 37169 35790 152 392 0 36768 0
Current children cumulated CPU time (s) 1800
Current children cumulated vsize (KiB) 148676

[startup+1802.2 s]
/proc/loadavg: 2.22 2.22 2.20 2/177 28465
/proc/meminfo: memFree=29790912/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=148676 CPUtime=1800 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) S 28279 28282 27985 0 -1 4194304 38538 0 0 0 179994 6 0 0 20 0 1 0 28684958 152244224 35790 33554432000 4194304 5796924 140734928512672 140734928506056 5190702 0 16384 4096 17954 18446744071579510697 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 37169 35790 152 392 0 36768 0
Current children cumulated CPU time (s) 1800
Current children cumulated vsize (KiB) 148676

[startup+1802.3 s]
/proc/loadavg: 2.22 2.22 2.20 2/177 28465
/proc/meminfo: memFree=29790912/32872528 swapFree=1459628/1655852
[pid=28282] ppid=28279 vsize=148676 CPUtime=1800 cores=1,3,5,7
/proc/28282/stat : 28282 (naps-1.02) S 28279 28282 27985 0 -1 4194304 38538 0 0 0 179994 6 0 0 20 0 1 0 28684958 152244224 35790 33554432000 4194304 5796924 140734928512672 140734928506056 5190702 0 16384 4096 17954 18446744071579510697 0 0 17 3 0 0 0 0 0
/proc/28282/statm: 37169 35790 152 392 0 36768 0
Current children cumulated CPU time (s) 1800
Current children cumulated vsize (KiB) 148676

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1802.3
CPU time (s): 1800.02
CPU user time (s): 1799.95
CPU system time (s): 0.072988
CPU usage (%): 99.8732
Max. virtual memory (cumulated for all children) (KiB): 148676

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1799.95
system time used= 0.072988
maximum resident set size= 143160
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 38538
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 25
involuntary context switches= 1853

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 5.03223 second user time and 7.72483 second system time

The end

Launcher Data 

Begin job on node136 at 2016-06-18 18:13:14
IDJOB=4103775
IDBENCH=2457
IDSOLVER=2544
FILE ID=node136/4103775-1466266394
RUNJOBID= node136-1466264583-28002
PBS_JOBID= 19918222
Free space on /tmp= 71728 MiB

SOLVER NAME= NaPS 1.02
BENCH NAME= PB06//final/normalized-PB06/OPT-BIGINT/mps-v2-20-10/ftp.netlib.org/lp/data/normalized-mps-v2-20-10-agg.opb
COMMAND LINE= naps-1.02 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4103775-1466266394/watcher-4103775-1466266394 -o /tmp/evaluation-result-4103775-1466266394/solver-4103775-1466266394 -C 1800 -W 1900 -M 15500 --bin-var /tmp/runsolver-exch-node136-1466264583-28002 --watchdog 1960  naps-1.02 HOME/instance-4103775-1466266394.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 4f19aa044e880280faec0077a2509ba8
RANDOM SEED=2060161683

node136.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.278
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.278
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.278
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.278
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.278
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.278
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.278
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.278
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32872528 kB
MemFree:        30153668 kB
Buffers:           32596 kB
Cached:           143896 kB
SwapCached:        47324 kB
Active:           196588 kB
Inactive:         160852 kB
Active(anon):     135796 kB
Inactive(anon):    47248 kB
Active(file):      60792 kB
Inactive(file):   113604 kB
Unevictable:     1655852 kB
Mlocked:         1459628 kB
SwapTotal:      67108856 kB
SwapFree:       67054628 kB
Dirty:             16096 kB
Writeback:             0 kB
AnonPages:       1830116 kB
Mapped:             9324 kB
Shmem:                 0 kB
Slab:             571416 kB
SReclaimable:      14940 kB
SUnreclaim:       556476 kB
KernelStack:        1472 kB
PageTables:         7396 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545120 kB
Committed_AS:    1912896 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346412 kB
VmallocChunk:   34341988032 kB
HardwareCorrupted:     0 kB
AnonHugePages:    237568 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7552 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 71728 MiB
End job on node136 at 2016-06-18 18:43:17