Trace number 2697646

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
pb_cplex 2010-07-01? (TO) 1800.72 643.117

General information on the benchmark

Name/SOFT-SMALLINT-LIN/PB06/web/uclid_pb_benchmarks/
normalized-dlx1c.ucl--soft-0-100-0.wbo
MD5SUM25ae7044365e399485d04cd072582b8a
Bench CategorySOFT-SMALLINT-LIN (only soft constraints, small integers, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark1
Best CPU time to get the best result obtained on this benchmark0.148976
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables6392
Total number of constraints18058
Number of soft constraints18058
Number of constraints which are clauses17278
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints780
Minimum length of a constraint1
Maximum length of a constraint11
Top cost 911376
Min constraint cost 1
Max constraint cost 100
Sum of constraints costs 911375
Biggest number in a constraint 33
Number of bits of the biggest number in a constraint 6
Biggest sum of numbers in a constraint 126
Number of bits of the biggest sum of numbers7
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c pb_cplex 1.0 - 20100701
0.00/0.00	c Parsing wbo file format. File HOME/instance-2697646-1278055925.wbo.
0.00/0.00	c Instance file HOME/instance-2697646-1278055925.wbo
0.00/0.00	c File size is 648975 bytes.
64.25/77.16	c Highest Coefficient sum: 126
64.25/77.16	c  IBM ILOG License Manager: "IBM ILOG Optimization Suite for Academic Initiative" is accessing CPLEX 12 with option(s): "e m b q ".
64.31/77.26	c ============================[ Problem Statistics ]=============================
64.31/77.26	c |                                                                             |
64.31/77.26	c |  Parsing time:          64.31        s                                      |
64.31/77.26	c |  Number Variables:      24450                                               |
64.33/77.34	c |  Number Constraints:    18059                                               |
64.33/77.34	c ===============================================================================
1800.72/643.11	c Remaining time 1735.69 sec.
1800.72/643.11	c 
1800.72/643.11	c *** INTERRUPTED ***
1800.72/643.11	s UNKNOWN
1800.72/643.11	c Total CPU time              : 1794.4 s
1800.72/643.11	c Memory used           : 352.02 MB
1800.72/643.11	c CPU time              : 1794.4 s
1800.72/643.11	c 
1800.72/643.11	c *** INTERRUPTED ***

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver version 3.2.9 (svn:492) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-2697646-1278055925/watcher-2697646-1278055925 -o /tmp/evaluation-result-2697646-1278055925/solver-2697646-1278055925 -C 1800 -W 2000 -M 3800 run HOME/instance-2697646-1278055925.wbo 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 3891200 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 3942400 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 12.34 10.50 6.77 11/187 21326
/proc/meminfo: memFree=31282112/32951124 swapFree=67111528/67111528
[pid=21325] ppid=21323 vsize=8700 CPUtime=0
/proc/21325/stat : 21325 (run) S 21323 21325 20647 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 146888615 8908800 248 4089446400 4194304 4922060 140736254261776 18446744073709551615 210204466309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/21325/statm: 2175 248 208 178 0 68 0
[pid=21326] ppid=21325 vsize=8700 CPUtime=0
/proc/21326/stat : 21326 (run) R 21325 21325 20647 0 -1 4202560 0 0 0 0 0 0 0 0 23 0 1 0 146888615 8908800 43 4089446400 4194304 4922060 140736254261776 18446744073709551615 210204467605 0 65538 4100 65536 0 0 0 17 0 0 0 0
/proc/21326/statm: 2175 43 3 178 0 68 0

[startup+0.0601661 s]
/proc/loadavg: 12.34 10.50 6.77 11/187 21326
/proc/meminfo: memFree=31282112/32951124 swapFree=67111528/67111528
[pid=21325] ppid=21323 vsize=8700 CPUtime=0
/proc/21325/stat : 21325 (run) S 21323 21325 20647 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 146888615 8908800 248 4089446400 4194304 4922060 140736254261776 18446744073709551615 210204466309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/21325/statm: 2175 248 208 178 0 68 0
[pid=21326] ppid=21325 vsize=13504 CPUtime=0.05
/proc/21326/stat : 21326 (pb_cplex) R 21325 21325 20647 0 -1 4202496 447 0 0 0 5 0 0 0 24 0 1 0 146888615 13828096 400 4089446400 134512640 146736320 4293689056 18446744073709551615 140648069 0 0 4096 16387 0 0 0 17 3 0 0 0
/proc/21326/statm: 3376 400 290 2985 0 151 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 22204

[startup+0.101158 s]
/proc/loadavg: 12.34 10.50 6.77 11/187 21326
/proc/meminfo: memFree=31282112/32951124 swapFree=67111528/67111528
[pid=21325] ppid=21323 vsize=8700 CPUtime=0
/proc/21325/stat : 21325 (run) S 21323 21325 20647 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 146888615 8908800 248 4089446400 4194304 4922060 140736254261776 18446744073709551615 210204466309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/21325/statm: 2175 248 208 178 0 68 0
[pid=21326] ppid=21325 vsize=13660 CPUtime=0.09
/proc/21326/stat : 21326 (pb_cplex) R 21325 21325 20647 0 -1 4202496 470 0 0 0 9 0 0 0 24 0 1 0 146888615 13987840 423 4089446400 134512640 146736320 4293689056 18446744073709551615 140681219 0 0 4096 16387 0 0 0 17 3 0 0 0
/proc/21326/statm: 3415 423 291 2985 0 190 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 22360

[startup+0.304158 s]
/proc/loadavg: 12.34 10.50 6.77 11/187 21326
/proc/meminfo: memFree=31282112/32951124 swapFree=67111528/67111528
[pid=21325] ppid=21323 vsize=8700 CPUtime=0
/proc/21325/stat : 21325 (run) S 21323 21325 20647 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 146888615 8908800 248 4089446400 4194304 4922060 140736254261776 18446744073709551615 210204466309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/21325/statm: 2175 248 208 178 0 68 0
[pid=21326] ppid=21325 vsize=13956 CPUtime=0.27
/proc/21326/stat : 21326 (pb_cplex) R 21325 21325 20647 0 -1 4202496 541 0 0 0 27 0 0 0 25 0 1 0 146888615 14290944 494 4089446400 134512640 146736320 4293689056 18446744073709551615 140679008 0 0 4096 16387 0 0 0 17 4 0 0 0
/proc/21326/statm: 3489 494 295 2985 0 264 0
Current children cumulated CPU time (s) 0.27
Current children cumulated vsize (KiB) 22656

[startup+0.726056 s]
/proc/loadavg: 12.34 10.50 6.77 11/187 21326
/proc/meminfo: memFree=31282112/32951124 swapFree=67111528/67111528
[pid=21325] ppid=21323 vsize=8700 CPUtime=0
/proc/21325/stat : 21325 (run) S 21323 21325 20647 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 146888615 8908800 248 4089446400 4194304 4922060 140736254261776 18446744073709551615 210204466309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/21325/statm: 2175 248 208 178 0 68 0
[pid=21326] ppid=21325 vsize=14128 CPUtime=0.46
/proc/21326/stat : 21326 (pb_cplex) R 21325 21325 20647 0 -1 4202496 585 0 0 0 46 0 0 0 25 0 1 0 146888615 14467072 538 4089446400 134512640 146736320 4293689056 18446744073709551615 140679059 0 0 4096 16387 0 0 0 17 5 0 0 0
/proc/21326/statm: 3532 538 298 2985 0 307 0
Current children cumulated CPU time (s) 0.46
Current children cumulated vsize (KiB) 22828

[startup+1.54992 s]
/proc/loadavg: 12.23 10.51 6.79 10/188 21327
/proc/meminfo: memFree=31280616/32951124 swapFree=67111528/67111528
[pid=21325] ppid=21323 vsize=8700 CPUtime=0
/proc/21325/stat : 21325 (run) S 21323 21325 20647 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 146888615 8908800 248 4089446400 4194304 4922060 140736254261776 18446744073709551615 210204466309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/21325/statm: 2175 248 208 178 0 68 0
[pid=21326] ppid=21325 vsize=14404 CPUtime=0.97
/proc/21326/stat : 21326 (pb_cplex) R 21325 21325 20647 0 -1 4202496 673 0 0 0 97 0 0 0 25 0 1 0 146888615 14749696 626 4089446400 134512640 146736320 4293689056 18446744073709551615 140684769 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/21326/statm: 3601 626 303 2985 0 376 0
Current children cumulated CPU time (s) 0.97
Current children cumulated vsize (KiB) 23104

[startup+3.10566 s]
/proc/loadavg: 12.23 10.51 6.79 3/188 21327
/proc/meminfo: memFree=31279748/32951124 swapFree=67111528/67111528
[pid=21325] ppid=21323 vsize=8700 CPUtime=0
/proc/21325/stat : 21325 (run) S 21323 21325 20647 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 146888615 8908800 248 4089446400 4194304 4922060 140736254261776 18446744073709551615 210204466309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/21325/statm: 2175 248 208 178 0 68 0
[pid=21326] ppid=21325 vsize=15108 CPUtime=2.12
/proc/21326/stat : 21326 (pb_cplex) R 21325 21325 20647 0 -1 4202496 824 0 0 0 212 0 0 0 25 0 1 0 146888615 15470592 777 4089446400 134512640 146736320 4293689056 18446744073709551615 141014496 0 0 4096 16387 0 0 0 17 3 0 0 0
/proc/21326/statm: 3777 777 311 2985 0 552 0
Current children cumulated CPU time (s) 2.12
Current children cumulated vsize (KiB) 23808
heavy processes:

[startup+6.30011 s]
/proc/loadavg: 12.23 10.51 6.79 3/188 21327
/proc/meminfo: memFree=31277888/32951124 swapFree=67111528/67111528
[pid=21325] ppid=21323 vsize=8700 CPUtime=0
/proc/21325/stat : 21325 (run) S 21323 21325 20647 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 146888615 8908800 248 4089446400 4194304 4922060 140736254261776 18446744073709551615 210204466309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/21325/statm: 2175 248 208 178 0 68 0
[pid=21326] ppid=21325 vsize=15692 CPUtime=4.24
/proc/21326/stat : 21326 (pb_cplex) R 21325 21325 20647 0 -1 4202496 996 0 0 0 424 0 0 0 25 0 1 0 146888615 16068608 949 4089446400 134512640 146736320 4293689056 18446744073709551615 140528792 0 0 4096 16387 0 0 0 17 5 0 0 0
/proc/21326/statm: 3923 949 322 2985 0 698 0
Current children cumulated CPU time (s) 4.24
Current children cumulated vsize (KiB) 24392
heavy processes:

[startup+12.738 s]
/proc/loadavg: 11.40 10.39 6.79 12/188 21327
/proc/meminfo: memFree=31277540/32951124 swapFree=67111528/67111528
[pid=21325] ppid=21323 vsize=8700 CPUtime=0
/proc/21325/stat : 21325 (run) S 21323 21325 20647 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 146888615 8908800 248 4089446400 4194304 4922060 140736254261776 18446744073709551615 210204466309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/21325/statm: 2175 248 208 178 0 68 0
[pid=21326] ppid=21325 vsize=16900 CPUtime=9.09
/proc/21326/stat : 21326 (pb_cplex) R 21325 21325 20647 0 -1 4202496 1280 0 0 0 909 0 0 0 25 0 1 0 146888615 17305600 1233 4089446400 134512640 146736320 4293689056 18446744073709551615 140683317 0 0 4096 16387 0 0 0 17 5 0 0 0
/proc/21326/statm: 4225 1233 340 2985 0 1000 0
Current children cumulated CPU time (s) 9.09
Current children cumulated vsize (KiB) 25600
heavy processes:

[startup+25.5489 s]
/proc/loadavg: 10.43 10.21 6.77 10/188 21327
/proc/meminfo: memFree=31276052/32951124 swapFree=67111528/67111528
[pid=21325] ppid=21323 vsize=8700 CPUtime=0
/proc/21325/stat : 21325 (run) S 21323 21325 20647 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 146888615 8908800 248 4089446400 4194304 4922060 140736254261776 18446744073709551615 210204466309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/21325/statm: 2175 248 208 178 0 68 0
[pid=21326] ppid=21325 vsize=18160 CPUtime=18.25
/proc/21326/stat : 21326 (pb_cplex) R 21325 21325 20647 0 -1 4202496 1662 0 0 0 1824 1 0 0 25 0 1 0 146888615 18595840 1615 4089446400 134512640 146736320 4293689056 18446744073709551615 140681199 0 0 4096 16387 0 0 0 17 3 0 0 0
/proc/21326/statm: 4540 1615 362 2985 0 1315 0
Current children cumulated CPU time (s) 18.25
Current children cumulated vsize (KiB) 26860
heavy processes:

[startup+51.1006 s]
/proc/loadavg: 8.56 9.79 6.74 3/188 21327
/proc/meminfo: memFree=31274888/32951124 swapFree=67111528/67111528
[pid=21325] ppid=21323 vsize=8700 CPUtime=0
/proc/21325/stat : 21325 (run) S 21323 21325 20647 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 146888615 8908800 248 4089446400 4194304 4922060 140736254261776 18446744073709551615 210204466309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/21325/statm: 2175 248 208 178 0 68 0
[pid=21326] ppid=21325 vsize=20784 CPUtime=41.03
/proc/21326/stat : 21326 (pb_cplex) R 21325 21325 20647 0 -1 4202496 2334 0 0 0 4101 2 0 0 25 0 1 0 146888615 21282816 2254 4089446400 134512640 146736320 4293689056 18446744073709551615 140714076 0 0 4096 16387 0 0 0 17 6 0 0 0
/proc/21326/statm: 5196 2254 401 2985 0 1971 0
Current children cumulated CPU time (s) 41.03
Current children cumulated vsize (KiB) 29484

[startup+102.301 s]
/proc/loadavg: 9.99 9.90 6.93 14/195 21336
/proc/meminfo: memFree=31010980/32951124 swapFree=67111528/67111528
[pid=21325] ppid=21323 vsize=8700 CPUtime=0
/proc/21325/stat : 21325 (run) S 21323 21325 20647 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 146888615 8908800 248 4089446400 4194304 4922060 140736254261776 18446744073709551615 210204466309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/21325/statm: 2175 248 208 178 0 68 0
[pid=21326] ppid=21325 vsize=459168 CPUtime=102.87
/proc/21326/stat : 21326 (pb_cplex) R 21325 21325 20647 0 -1 4202496 209943 0 0 0 10245 42 0 0 20 0 8 0 146888615 470188032 67750 4089446400 134512640 146736320 4293689056 18446744073709551615 136400566 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/21326/statm: 114792 67756 1082 2985 0 111334 0
[pid=21326/tid=21330] ppid=21325 vsize=459168 CPUtime=3.66
/proc/21326/task/21330/stat : 21330 (pb_cplex) R 21325 21325 20647 0 -1 4202560 17384 0 0 0 363 3 0 0 18 0 8 0 146897366 470188032 67786 4089446400 134512640 146736320 4293689056 18446744073709551615 139042910 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=21326/tid=21331] ppid=21325 vsize=459168 CPUtime=3.46
/proc/21326/task/21331/stat : 21331 (pb_cplex) R 21325 21325 20647 0 -1 4202560 14216 0 0 0 344 2 0 0 18 0 8 0 146897366 470188032 67791 4089446400 134512640 146736320 4293689056 18446744073709551615 136017485 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=21326/tid=21332] ppid=21325 vsize=459168 CPUtime=3.55
/proc/21326/task/21332/stat : 21332 (pb_cplex) R 21325 21325 20647 0 -1 4202560 16445 0 0 0 352 3 0 0 18 0 8 0 146897366 470188032 67791 4089446400 134512640 146736320 4293689056 18446744073709551615 139044510 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=21326/tid=21333] ppid=21325 vsize=459168 CPUtime=3.63
/proc/21326/task/21333/stat : 21333 (pb_cplex) S 21325 21325 20647 0 -1 4202560 12158 0 0 0 361 2 0 0 18 0 8 0 146897366 470188032 67791 4089446400 134512640 146736320 4293689056 18446744073709551615 144580073 0 2 4096 16387 18446604470103206656 0 0 -1 4 0 0 0
[pid=21326/tid=21334] ppid=21325 vsize=459168 CPUtime=2.78
/proc/21326/task/21334/stat : 21334 (pb_cplex) S 21325 21325 20647 0 -1 4202560 13260 0 0 0 275 3 0 0 18 0 8 0 146897366 470188032 67791 4089446400 134512640 146736320 4293689056 18446744073709551615 144580073 0 2 4096 16387 18446604470103206656 0 0 -1 5 0 0 0
[pid=21326/tid=21335] ppid=21325 vsize=459168 CPUtime=2.8
/proc/21326/task/21335/stat : 21335 (pb_cplex) R 21325 21325 20647 0 -1 4202560 11435 0 0 0 278 2 0 0 18 0 8 0 146897366 470188032 67791 4089446400 134512640 146736320 4293689056 18446744073709551615 136007588 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=21326/tid=21336] ppid=21325 vsize=459168 CPUtime=4.16
/proc/21326/task/21336/stat : 21336 (pb_cplex) R 21325 21325 20647 0 -1 4202560 17385 0 0 0 413 3 0 0 20 0 8 0 146897366 470188032 67791 4089446400 134512640 146736320 4293689056 18446744073709551615 137045143 0 2 4096 16387 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 102.87
Current children cumulated vsize (KiB) 467868

[startup+162.326 s]
/proc/loadavg: 10.66 10.21 7.23 3/188 21338
/proc/meminfo: memFree=31076208/32951124 swapFree=67111528/67111528
[pid=21325] ppid=21323 vsize=8700 CPUtime=0
/proc/21325/stat : 21325 (run) S 21323 21325 20647 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 146888615 8908800 248 4089446400 4194304 4922060 140736254261776 18446744073709551615 210204466309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/21325/statm: 2175 248 208 178 0 68 0
[pid=21326] ppid=21325 vsize=242700 CPUtime=228.89
/proc/21326/stat : 21326 (pb_cplex) R 21325 21325 20647 0 -1 4202496 384997 0 0 0 22799 90 0 0 18 0 1 0 146888615 248524800 38827 4089446400 134512640 146736320 4293689056 18446744073709551615 144757825 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/21326/statm: 60675 38827 1074 2985 0 57217 0
Current children cumulated CPU time (s) 228.89
Current children cumulated vsize (KiB) 251400

[startup+222.301 s]
/proc/loadavg: 8.67 9.67 7.22 6/195 21354
/proc/meminfo: memFree=31012164/32951124 swapFree=67111528/67111528
[pid=21325] ppid=21323 vsize=8700 CPUtime=0
/proc/21325/stat : 21325 (run) S 21323 21325 20647 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 146888615 8908800 248 4089446400 4194304 4922060 140736254261776 18446744073709551615 210204466309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/21325/statm: 2175 248 208 178 0 68 0
[pid=21326] ppid=21325 vsize=403436 CPUtime=289.99
/proc/21326/stat : 21326 (pb_cplex) R 21325 21325 20647 0 -1 4202496 464838 0 0 0 28890 109 0 0 25 0 8 0 146888615 413118464 68289 4089446400 134512640 146736320 4293689056 18446744073709551615 136007277 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0
/proc/21326/statm: 100859 68289 1092 2985 0 97401 0
[pid=21326/tid=21348] ppid=21325 vsize=403436 CPUtime=2
/proc/21326/task/21348/stat : 21348 (pb_cplex) R 21325 21325 20647 0 -1 4202560 4397 0 0 0 200 0 0 0 21 0 8 0 146910254 413118464 68289 4089446400 134512640 146736320 4293689056 18446744073709551615 135850298 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=21326/tid=21349] ppid=21325 vsize=403436 CPUtime=0.32
/proc/21326/task/21349/stat : 21349 (pb_cplex) S 21325 21325 20647 0 -1 4202560 6246 0 0 0 31 1 0 0 16 0 8 0 146910254 413118464 68289 4089446400 134512640 146736320 4293689056 18446744073709551615 144580073 0 2 4096 16387 18446604470103206656 0 0 -1 3 0 0 0
[pid=21326/tid=21350] ppid=21325 vsize=403436 CPUtime=2.05
/proc/21326/task/21350/stat : 21350 (pb_cplex) R 21325 21325 20647 0 -1 4202560 5450 0 0 0 204 1 0 0 21 0 8 0 146910254 413118464 68289 4089446400 134512640 146736320 4293689056 18446744073709551615 136007616 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=21326/tid=21351] ppid=21325 vsize=403436 CPUtime=1.28

################
# More data... #
################

[pid=21326] ppid=21325 vsize=445236 CPUtime=728.12
/proc/21326/stat : 21326 (pb_cplex) R 21325 21325 20647 0 -1 4202496 1198861 0 0 0 72531 281 0 0 15 0 8 0 146888615 455921664 85351 4089446400 134512640 146736320 4293689056 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/21326/statm: 111309 85351 1084 2985 0 107851 0
[pid=21326/tid=21348] ppid=21325 vsize=445236 CPUtime=50.6
/proc/21326/task/21348/stat : 21348 (pb_cplex) R 21325 21325 20647 0 -1 4202560 75735 0 0 0 5041 19 0 0 18 0 8 0 146910254 455921664 85351 4089446400 134512640 146736320 4293689056 18446744073709551615 136008234 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=21326/tid=21349] ppid=21325 vsize=445236 CPUtime=61.23
/proc/21326/task/21349/stat : 21349 (pb_cplex) R 21325 21325 20647 0 -1 4202560 116289 0 0 0 6098 25 0 0 23 0 8 0 146910254 455921664 85351 4089446400 134512640 146736320 4293689056 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=21326/tid=21350] ppid=21325 vsize=445236 CPUtime=54.03
/proc/21326/task/21350/stat : 21350 (pb_cplex) R 21325 21325 20647 0 -1 4202560 117997 0 0 0 5378 25 0 0 15 0 8 0 146910254 455921664 85351 4089446400 134512640 146736320 4293689056 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=21326/tid=21351] ppid=21325 vsize=445236 CPUtime=59.23
/proc/21326/task/21351/stat : 21351 (pb_cplex) R 21325 21325 20647 0 -1 4202560 77832 0 0 0 5906 17 0 0 16 0 8 0 146910254 455921664 85351 4089446400 134512640 146736320 4293689056 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=21326/tid=21352] ppid=21325 vsize=445236 CPUtime=54.06
/proc/21326/task/21352/stat : 21352 (pb_cplex) R 21325 21325 20647 0 -1 4202560 95824 0 0 0 5383 23 0 0 16 0 8 0 146910254 455921664 85351 4089446400 134512640 146736320 4293689056 18446744073709551615 135526849 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=21326/tid=21353] ppid=21325 vsize=445236 CPUtime=56.22
/proc/21326/task/21353/stat : 21353 (pb_cplex) R 21325 21325 20647 0 -1 4202560 121889 0 0 0 5596 26 0 0 18 0 8 0 146910254 455921664 85351 4089446400 134512640 146736320 4293689056 18446744073709551615 135999265 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=21326/tid=21354] ppid=21325 vsize=445236 CPUtime=60.79
/proc/21326/task/21354/stat : 21354 (pb_cplex) R 21325 21325 20647 0 -1 4202560 110866 0 0 0 6053 26 0 0 20 0 8 0 146910254 455921664 85351 4089446400 134512640 146736320 4293689056 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 728.12
Current children cumulated vsize (KiB) 453936

[startup+402.301 s]
/proc/loadavg: 7.65 9.28 7.56 2/179 21387
/proc/meminfo: memFree=30981312/32951124 swapFree=67111528/67111528
[pid=21325] ppid=21323 vsize=8700 CPUtime=0
/proc/21325/stat : 21325 (run) S 21323 21325 20647 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 146888615 8908800 248 4089446400 4194304 4922060 140736254261776 18446744073709551615 210204466309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/21325/statm: 2175 248 208 178 0 68 0
[pid=21326] ppid=21325 vsize=481256 CPUtime=1126.89
/proc/21326/stat : 21326 (pb_cplex) R 21325 21325 20647 0 -1 4202496 2077337 0 0 0 112204 485 0 0 16 0 8 0 146888615 492806144 92227 4089446400 134512640 146736320 4293689056 18446744073709551615 136400430 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0
/proc/21326/statm: 120314 92227 1085 2985 0 116856 0
[pid=21326/tid=21348] ppid=21325 vsize=481256 CPUtime=100.76
/proc/21326/task/21348/stat : 21348 (pb_cplex) R 21325 21325 20647 0 -1 4202560 163101 0 0 0 10034 42 0 0 23 0 8 0 146910254 492806144 92227 4089446400 134512640 146736320 4293689056 18446744073709551615 136007053 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=21326/tid=21349] ppid=21325 vsize=481256 CPUtime=110.32
/proc/21326/task/21349/stat : 21349 (pb_cplex) R 21325 21325 20647 0 -1 4202560 230501 0 0 0 10980 52 0 0 16 0 8 0 146910254 492806144 92227 4089446400 134512640 146736320 4293689056 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=21326/tid=21350] ppid=21325 vsize=481256 CPUtime=102.64
/proc/21326/task/21350/stat : 21350 (pb_cplex) R 21325 21325 20647 0 -1 4202560 230862 0 0 0 10213 51 0 0 16 0 8 0 146910254 492806144 92227 4089446400 134512640 146736320 4293689056 18446744073709551615 137366915 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=21326/tid=21351] ppid=21325 vsize=481256 CPUtime=108.78
/proc/21326/task/21351/stat : 21351 (pb_cplex) R 21325 21325 20647 0 -1 4202560 175246 0 0 0 10838 40 0 0 16 0 8 0 146910254 492806144 92227 4089446400 134512640 146736320 4293689056 18446744073709551615 136008165 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=21326/tid=21352] ppid=21325 vsize=481256 CPUtime=103.48
/proc/21326/task/21352/stat : 21352 (pb_cplex) R 21325 21325 20647 0 -1 4202560 200511 0 0 0 10299 49 0 0 16 0 8 0 146910254 492806144 92227 4089446400 134512640 146736320 4293689056 18446744073709551615 136007030 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=21326/tid=21353] ppid=21325 vsize=481256 CPUtime=106.61
/proc/21326/task/21353/stat : 21353 (pb_cplex) R 21325 21325 20647 0 -1 4202560 233816 0 0 0 10610 51 0 0 16 0 8 0 146910254 492806144 92227 4089446400 134512640 146736320 4293689056 18446744073709551615 136007371 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=21326/tid=21354] ppid=21325 vsize=481256 CPUtime=110.58
/proc/21326/task/21354/stat : 21354 (pb_cplex) R 21325 21325 20647 0 -1 4202560 241300 0 0 0 11004 54 0 0 18 0 8 0 146910254 492806144 92227 4089446400 134512640 146736320 4293689056 18446744073709551615 136017748 0 2 4096 16387 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 1126.89
Current children cumulated vsize (KiB) 489956

[startup+462.306 s]
/proc/loadavg: 4.90 8.33 7.34 2/172 21389
/proc/meminfo: memFree=31131540/32951124 swapFree=67111528/67111528
[pid=21325] ppid=21323 vsize=8700 CPUtime=0
/proc/21325/stat : 21325 (run) S 21323 21325 20647 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 146888615 8908800 248 4089446400 4194304 4922060 140736254261776 18446744073709551615 210204466309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/21325/statm: 2175 248 208 178 0 68 0
[pid=21326] ppid=21325 vsize=283800 CPUtime=1322.35
/proc/21326/stat : 21326 (pb_cplex) R 21325 21325 20647 0 -1 4202496 2549026 0 0 0 131615 620 0 0 25 0 1 0 146888615 290611200 56723 4089446400 134512640 146736320 4293689056 18446744073709551615 136000075 0 0 4096 16387 0 0 0 17 7 0 0 0
/proc/21326/statm: 70950 56723 1086 2985 0 67492 0
Current children cumulated CPU time (s) 1322.35
Current children cumulated vsize (KiB) 292500

[startup+522.301 s]
/proc/loadavg: 2.43 6.99 6.94 2/172 21391
/proc/meminfo: memFree=31133364/32951124 swapFree=67111528/67111528
[pid=21325] ppid=21323 vsize=8700 CPUtime=0
/proc/21325/stat : 21325 (run) S 21323 21325 20647 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 146888615 8908800 248 4089446400 4194304 4922060 140736254261776 18446744073709551615 210204466309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/21325/statm: 2175 248 208 178 0 68 0
[pid=21326] ppid=21325 vsize=283800 CPUtime=1382.35
/proc/21326/stat : 21326 (pb_cplex) R 21325 21325 20647 0 -1 4202496 2549026 0 0 0 137614 621 0 0 25 0 1 0 146888615 290611200 56723 4089446400 134512640 146736320 4293689056 18446744073709551615 135525920 0 0 4096 16387 0 0 0 17 7 0 0 0
/proc/21326/statm: 70950 56723 1086 2985 0 67492 0
Current children cumulated CPU time (s) 1382.35
Current children cumulated vsize (KiB) 292500

[startup+582.301 s]
/proc/loadavg: 3.22 6.27 6.69 9/179 21398
/proc/meminfo: memFree=31103980/32951124 swapFree=67111528/67111528
[pid=21325] ppid=21323 vsize=8700 CPUtime=0
/proc/21325/stat : 21325 (run) S 21323 21325 20647 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 146888615 8908800 248 4089446400 4194304 4922060 140736254261776 18446744073709551615 210204466309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/21325/statm: 2175 248 208 178 0 68 0
[pid=21326] ppid=21325 vsize=345112 CPUtime=1536.86
/proc/21326/stat : 21326 (pb_cplex) R 21325 21325 20647 0 -1 4202496 2568302 0 0 0 153059 627 0 0 22 0 8 0 146888615 353394688 63532 4089446400 134512640 146736320 4293689056 18446744073709551615 135524905 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0
/proc/21326/statm: 86278 63532 1087 2985 0 82820 0
[pid=21326/tid=21392] ppid=21325 vsize=345112 CPUtime=13.5
/proc/21326/task/21392/stat : 21392 (pb_cplex) R 21325 21325 20647 0 -1 4202560 489 0 0 0 1350 0 0 0 25 0 8 0 146944201 353394688 63532 4089446400 134512640 146736320 4293689056 18446744073709551615 137342636 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=21326/tid=21393] ppid=21325 vsize=345112 CPUtime=13.51
/proc/21326/task/21393/stat : 21393 (pb_cplex) R 21325 21325 20647 0 -1 4202560 402 0 0 0 1351 0 0 0 25 0 8 0 146944201 353394688 63532 4089446400 134512640 146736320 4293689056 18446744073709551615 136000176 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=21326/tid=21394] ppid=21325 vsize=345112 CPUtime=13.51
/proc/21326/task/21394/stat : 21394 (pb_cplex) R 21325 21325 20647 0 -1 4202560 1084 0 0 0 1351 0 0 0 25 0 8 0 146944201 353394688 63532 4089446400 134512640 146736320 4293689056 18446744073709551615 136388073 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=21326/tid=21395] ppid=21325 vsize=345112 CPUtime=13.51
/proc/21326/task/21395/stat : 21395 (pb_cplex) R 21325 21325 20647 0 -1 4202560 1577 0 0 0 1351 0 0 0 25 0 8 0 146944201 353394688 63532 4089446400 134512640 146736320 4293689056 18446744073709551615 136008050 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=21326/tid=21396] ppid=21325 vsize=345112 CPUtime=13.51
/proc/21326/task/21396/stat : 21396 (pb_cplex) R 21325 21325 20647 0 -1 4202560 3133 0 0 0 1351 0 0 0 25 0 8 0 146944201 353394688 63532 4089446400 134512640 146736320 4293689056 18446744073709551615 139042876 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=21326/tid=21397] ppid=21325 vsize=345112 CPUtime=13.43
/proc/21326/task/21397/stat : 21397 (pb_cplex) R 21325 21325 20647 0 -1 4202560 4380 0 0 0 1342 1 0 0 22 0 8 0 146944201 353394688 63532 4089446400 134512640 146736320 4293689056 18446744073709551615 136018936 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=21326/tid=21398] ppid=21325 vsize=345112 CPUtime=13.47
/proc/21326/task/21398/stat : 21398 (pb_cplex) R 21325 21325 20647 0 -1 4202560 1424 0 0 0 1347 0 0 0 25 0 8 0 146944201 353394688 63532 4089446400 134512640 146736320 4293689056 18446744073709551615 136008065 0 2 4096 16387 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1536.86
Current children cumulated vsize (KiB) 353812

[startup+642.301 s]
/proc/loadavg: 4.32 5.96 6.55 9/179 21400
/proc/meminfo: memFree=31092772/32951124 swapFree=67111528/67111528
[pid=21325] ppid=21323 vsize=8700 CPUtime=0
/proc/21325/stat : 21325 (run) S 21323 21325 20647 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 146888615 8908800 248 4089446400 4194304 4922060 140736254261776 18446744073709551615 210204466309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/21325/statm: 2175 248 208 178 0 68 0
[pid=21326] ppid=21325 vsize=360436 CPUtime=1794.28
/proc/21326/stat : 21326 (pb_cplex) R 21325 21325 20647 0 -1 4202496 2593001 0 0 0 178792 636 0 0 25 0 8 0 146888615 369086464 66802 4089446400 134512640 146736320 4293689056 18446744073709551615 135844359 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0
/proc/21326/statm: 90109 66802 1091 2985 0 86651 0
[pid=21326/tid=21392] ppid=21325 vsize=360436 CPUtime=37.87
/proc/21326/task/21392/stat : 21392 (pb_cplex) R 21325 21325 20647 0 -1 4202560 3808 0 0 0 3786 1 0 0 25 0 8 0 146944201 369086464 66802 4089446400 134512640 146736320 4293689056 18446744073709551615 136008065 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=21326/tid=21393] ppid=21325 vsize=360436 CPUtime=44.92
/proc/21326/task/21393/stat : 21393 (pb_cplex) R 21325 21325 20647 0 -1 4202560 6139 0 0 0 4490 2 0 0 25 0 8 0 146944201 369086464 66802 4089446400 134512640 146736320 4293689056 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=21326/tid=21394] ppid=21325 vsize=360436 CPUtime=34.12
/proc/21326/task/21394/stat : 21394 (pb_cplex) R 21325 21325 20647 0 -1 4202560 1841 0 0 0 3412 0 0 0 25 0 8 0 146944201 369086464 66802 4089446400 134512640 146736320 4293689056 18446744073709551615 136009167 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=21326/tid=21395] ppid=21325 vsize=360436 CPUtime=33.9
/proc/21326/task/21395/stat : 21395 (pb_cplex) R 21325 21325 20647 0 -1 4202560 1998 0 0 0 3390 0 0 0 25 0 8 0 146944201 369086464 66802 4089446400 134512640 146736320 4293689056 18446744073709551615 136001912 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=21326/tid=21396] ppid=21325 vsize=360436 CPUtime=47.61
/proc/21326/task/21396/stat : 21396 (pb_cplex) R 21325 21325 20647 0 -1 4202560 10504 0 0 0 4760 1 0 0 25 0 8 0 146944201 369086464 66802 4089446400 134512640 146736320 4293689056 18446744073709551615 139042974 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=21326/tid=21397] ppid=21325 vsize=360436 CPUtime=73.28
/proc/21326/task/21397/stat : 21397 (pb_cplex) R 21325 21325 20647 0 -1 4202560 6157 0 0 0 7326 2 0 0 25 0 8 0 146944201 369086464 66802 4089446400 134512640 146736320 4293689056 18446744073709551615 136008050 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=21326/tid=21398] ppid=21325 vsize=360436 CPUtime=50.24
/proc/21326/task/21398/stat : 21398 (pb_cplex) R 21325 21325 20647 0 -1 4202560 1993 0 0 0 5023 1 0 0 25 0 8 0 146944201 369086464 66802 4089446400 134512640 146736320 4293689056 18446744073709551615 136008069 0 2 4096 16387 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1794.28
Current children cumulated vsize (KiB) 369136



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+643.1 s]
/proc/loadavg: 4.32 5.96 6.55 9/179 21400
/proc/meminfo: memFree=31092896/32951124 swapFree=67111528/67111528
[pid=21325] ppid=21323 vsize=8700 CPUtime=0
/proc/21325/stat : 21325 (run) S 21323 21325 20647 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 146888615 8908800 248 4089446400 4194304 4922060 140736254261776 18446744073709551615 210204466309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/21325/statm: 2175 248 208 178 0 68 0
[pid=21326] ppid=21325 vsize=360468 CPUtime=1800.72
/proc/21326/stat : 21326 (pb_cplex) R 21325 21325 20647 0 -1 4202496 2593015 0 0 0 179436 636 0 0 25 0 8 0 146888615 369119232 66816 4089446400 134512640 146736320 4293689056 18446744073709551615 136475859 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0
/proc/21326/statm: 90117 66816 1091 2985 0 86659 0
[pid=21326/tid=21392] ppid=21325 vsize=360468 CPUtime=38.67
/proc/21326/task/21392/stat : 21392 (pb_cplex) R 21325 21325 20647 0 -1 4202560 3818 0 0 0 3866 1 0 0 25 0 8 0 146944201 369119232 66816 4089446400 134512640 146736320 4293689056 18446744073709551615 136008160 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=21326/tid=21393] ppid=21325 vsize=360468 CPUtime=45.73
/proc/21326/task/21393/stat : 21393 (pb_cplex) R 21325 21325 20647 0 -1 4202560 6139 0 0 0 4571 2 0 0 25 0 8 0 146944201 369119232 66816 4089446400 134512640 146736320 4293689056 18446744073709551615 136008042 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=21326/tid=21394] ppid=21325 vsize=360468 CPUtime=34.93
/proc/21326/task/21394/stat : 21394 (pb_cplex) R 21325 21325 20647 0 -1 4202560 1841 0 0 0 3493 0 0 0 25 0 8 0 146944201 369119232 66816 4089446400 134512640 146736320 4293689056 18446744073709551615 135454096 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=21326/tid=21395] ppid=21325 vsize=360468 CPUtime=34.71
/proc/21326/task/21395/stat : 21395 (pb_cplex) R 21325 21325 20647 0 -1 4202560 1998 0 0 0 3471 0 0 0 25 0 8 0 146944201 369119232 66816 4089446400 134512640 146736320 4293689056 18446744073709551615 136384857 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=21326/tid=21396] ppid=21325 vsize=360468 CPUtime=48.42
/proc/21326/task/21396/stat : 21396 (pb_cplex) R 21325 21325 20647 0 -1 4202560 10507 0 0 0 4841 1 0 0 25 0 8 0 146944201 369119232 66816 4089446400 134512640 146736320 4293689056 18446744073709551615 139044555 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=21326/tid=21397] ppid=21325 vsize=360468 CPUtime=74.07
/proc/21326/task/21397/stat : 21397 (pb_cplex) R 21325 21325 20647 0 -1 4202560 6157 0 0 0 7405 2 0 0 25 0 8 0 146944201 369119232 66816 4089446400 134512640 146736320 4293689056 18446744073709551615 136403109 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=21326/tid=21398] ppid=21325 vsize=360468 CPUtime=51.04
/proc/21326/task/21398/stat : 21398 (pb_cplex) R 21325 21325 20647 0 -1 4202560 1993 0 0 0 5103 1 0 0 25 0 8 0 146944201 369119232 66816 4089446400 134512640 146736320 4293689056 18446744073709551615 136008042 0 2 4096 16387 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1800.72
Current children cumulated vsize (KiB) 369168

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 21325 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=0
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=999
# CPU time returned by wait4() is 0.000999
# while last known CPU time is 1800.72
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 643.117
CPU time (s): 1800.72
CPU user time (s): 1794.36
CPU system time (s): 6.36
CPU usage (%): 279.999
Max. virtual memory (cumulated for all children) (KiB): 526160

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0
system time used= 0.000999
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 364
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2
involuntary context switches= 0

runsolver used 1.14682 second user time and 2.23866 second system time

The end

Launcher Data 

Begin job on node130 at 2010-07-02 09:32:05
IDJOB=2697646
IDBENCH=79325
IDSOLVER=1210
FILE ID=node130/2697646-1278055925
PBS_JOBID= 11198921
Free space on /tmp= 74168 MiB

SOLVER NAME= pb_cplex 2010-07-01
BENCH NAME= PB10/normalized-PB10/SOFT-SMALLINT-LIN/PB06/web/uclid_pb_benchmarks/normalized-dlx1c.ucl--soft-0-100-0.wbo
COMMAND LINE= run BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-2697646-1278055925/watcher-2697646-1278055925 -o /tmp/evaluation-result-2697646-1278055925/solver-2697646-1278055925 -C 1800 -W 2000 -M 3800  run HOME/instance-2697646-1278055925.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 3800 MiB
MAX NB THREAD= 0

MD5SUM BENCH= 25ae7044365e399485d04cd072582b8a
RANDOM SEED=2076072538

node130.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.796
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.796
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.796
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.796
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.72
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.796
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.796
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.796
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.796
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5237.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      31282648 kB
Buffers:        224092 kB
Cached:        1176620 kB
SwapCached:          0 kB
Active:         687180 kB
Inactive:       831376 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      31282648 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           26364 kB
Writeback:          24 kB
AnonPages:      117416 kB
Mapped:          19120 kB
Slab:            82908 kB
PageTables:       5112 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   339480 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 74168 MiB
End job on node130 at 2010-07-02 09:42:51