Trace number 2697508

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
pb_cplex 2010-07-01? (TO) 1800.37 720.217

General information on the benchmark

Name/PARTIAL-SMALLINT-LIN/PB06/web/uclid_pb_benchmarks/
normalized-ooo.tag8.ucl--soft-66-100-0.wbo
MD5SUMfbaddd9625623c5e0a4f0ddddf2cf090
Bench CategoryPARTIAL-SMALLINT-LIN (both soft and hard constraints, small integers, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark2
Best CPU time to get the best result obtained on this benchmark2.6076
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables3249
Total number of constraints8983
Number of soft constraints3029
Number of constraints which are clauses8599
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints384
Minimum length of a constraint1
Maximum length of a constraint11
Top cost 154751
Min constraint cost 1
Max constraint cost 100
Sum of constraints costs 154750
Biggest number in a constraint 33
Number of bits of the biggest number in a constraint 6
Biggest sum of numbers in a constraint 126
Number of bits of the biggest sum of numbers7
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c pb_cplex 1.0 - 20100701
0.00/0.00	c Parsing wbo file format. File HOME/instance-2697508-1278055359.wbo.
0.00/0.00	c Instance file HOME/instance-2697508-1278055359.wbo
0.00/0.00	c File size is 287136 bytes.
1.80/4.31	c Highest Coefficient sum: 126
1.80/4.31	c  IBM ILOG License Manager: "IBM ILOG Optimization Suite for Academic Initiative" is accessing CPLEX 12 with option(s): "e m b q ".
1.91/4.34	c ============================[ Problem Statistics ]=============================
1.91/4.34	c |                                                                             |
1.91/4.34	c |  Parsing time:          1.92         s                                      |
1.91/4.34	c |  Number Variables:      6278                                                |
1.91/4.34	c |  Number Constraints:    8984                                                |
1.91/4.34	c ===============================================================================
1800.37/720.21	c Remaining time 1798.08 sec.
1800.37/720.21	c 
1800.37/720.21	c *** INTERRUPTED ***
1800.37/720.21	s UNKNOWN
1800.37/720.21	c Total CPU time              : 1797.36 s
1800.37/720.21	c Memory used           : 325.32 MB
1800.37/720.21	c CPU time              : 1797.36 s
1800.37/720.21	c 
1800.37/720.21	c *** INTERRUPTED ***

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver version 3.2.9 (svn:492) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-2697508-1278055359/watcher-2697508-1278055359 -o /tmp/evaluation-result-2697508-1278055359/solver-2697508-1278055359 -C 1800 -W 2000 -M 3800 run HOME/instance-2697508-1278055359.wbo 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 3891200 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 3942400 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 9.46 7.58 3.82 4/186 22647
/proc/meminfo: memFree=31286776/32951124 swapFree=67111528/67111528
[pid=22647] ppid=22645 vsize=8700 CPUtime=0
/proc/22647/stat : 22647 (run) S 22645 22647 21865 0 -1 4202496 364 0 0 0 0 0 0 0 25 0 1 0 146831162 8908800 248 4089446400 4194304 4922060 140736532750384 18446744073709551615 270338202757 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/22647/statm: 2175 248 208 178 0 68 0

[startup+0.098594 s]
/proc/loadavg: 9.46 7.58 3.82 4/186 22647
/proc/meminfo: memFree=31286776/32951124 swapFree=67111528/67111528
[pid=22647] ppid=22645 vsize=8700 CPUtime=0
/proc/22647/stat : 22647 (run) S 22645 22647 21865 0 -1 4202496 364 0 0 0 0 0 0 0 25 0 1 0 146831162 8908800 248 4089446400 4194304 4922060 140736532750384 18446744073709551615 270338202757 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/22647/statm: 2175 248 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.101209 s]
/proc/loadavg: 9.46 7.58 3.82 4/186 22647
/proc/meminfo: memFree=31286776/32951124 swapFree=67111528/67111528
[pid=22647] ppid=22645 vsize=8700 CPUtime=0
/proc/22647/stat : 22647 (run) S 22645 22647 21865 0 -1 4202496 364 0 0 0 0 0 0 0 25 0 1 0 146831162 8908800 248 4089446400 4194304 4922060 140736532750384 18446744073709551615 270338202757 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/22647/statm: 2175 248 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.300138 s]
/proc/loadavg: 9.46 7.58 3.82 4/186 22647
/proc/meminfo: memFree=31286776/32951124 swapFree=67111528/67111528
[pid=22647] ppid=22645 vsize=8700 CPUtime=0
/proc/22647/stat : 22647 (run) S 22645 22647 21865 0 -1 4202496 364 0 0 0 0 0 0 0 25 0 1 0 146831162 8908800 248 4089446400 4194304 4922060 140736532750384 18446744073709551615 270338202757 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/22647/statm: 2175 248 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.702059 s]
/proc/loadavg: 9.46 7.58 3.82 4/186 22647
/proc/meminfo: memFree=31286776/32951124 swapFree=67111528/67111528
[pid=22647] ppid=22645 vsize=8700 CPUtime=0
/proc/22647/stat : 22647 (run) S 22645 22647 21865 0 -1 4202496 364 0 0 0 0 0 0 0 25 0 1 0 146831162 8908800 248 4089446400 4194304 4922060 140736532750384 18446744073709551615 270338202757 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/22647/statm: 2175 248 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+1.5009 s]
/proc/loadavg: 9.46 7.58 3.82 9/188 22649
/proc/meminfo: memFree=31284280/32951124 swapFree=67111528/67111528
[pid=22647] ppid=22645 vsize=8700 CPUtime=0
/proc/22647/stat : 22647 (run) S 22645 22647 21865 0 -1 4202496 364 0 0 0 0 0 0 0 25 0 1 0 146831162 8908800 248 4089446400 4194304 4922060 140736532750384 18446744073709551615 270338202757 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/22647/statm: 2175 248 208 178 0 68 0
[pid=22648] ppid=22647 vsize=14732 CPUtime=0.69
/proc/22648/stat : 22648 (pb_cplex) R 22647 22647 21865 0 -1 4202496 857 0 0 0 69 0 0 0 25 0 1 0 146831162 15085568 811 4089446400 134512640 146736320 4293752208 18446744073709551615 140408977 0 0 4096 16387 0 0 0 17 2 0 0 0
/proc/22648/statm: 3683 811 324 2985 0 546 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 23432

[startup+3.10158 s]
/proc/loadavg: 8.87 7.49 3.81 11/188 22649
/proc/meminfo: memFree=31283784/32951124 swapFree=67111528/67111528
[pid=22647] ppid=22645 vsize=8700 CPUtime=0
/proc/22647/stat : 22647 (run) S 22645 22647 21865 0 -1 4202496 364 0 0 0 0 0 0 0 25 0 1 0 146831162 8908800 248 4089446400 4194304 4922060 140736532750384 18446744073709551615 270338202757 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/22647/statm: 2175 248 208 178 0 68 0
[pid=22648] ppid=22647 vsize=15416 CPUtime=1.34
/proc/22648/stat : 22648 (pb_cplex) R 22647 22647 21865 0 -1 4202496 1054 0 0 0 134 0 0 0 25 0 1 0 146831162 15785984 1008 4089446400 134512640 146736320 4293752208 18446744073709551615 140679064 0 0 4096 16387 0 0 0 17 6 0 0 0
/proc/22648/statm: 3854 1008 340 2985 0 717 0
Current children cumulated CPU time (s) 1.34
Current children cumulated vsize (KiB) 24116
heavy processes:

[startup+6.30098 s]
/proc/loadavg: 8.87 7.49 3.81 3/188 22650
/proc/meminfo: memFree=31262580/32951124 swapFree=67111528/67111528
[pid=22647] ppid=22645 vsize=8700 CPUtime=0
/proc/22647/stat : 22647 (run) S 22645 22647 21865 0 -1 4202496 364 0 0 0 0 0 0 0 25 0 1 0 146831162 8908800 248 4089446400 4194304 4922060 140736532750384 18446744073709551615 270338202757 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/22647/statm: 2175 248 208 178 0 68 0
[pid=22648] ppid=22647 vsize=43908 CPUtime=3.75
/proc/22648/stat : 22648 (pb_cplex) R 22647 22647 21865 0 -1 4202496 17184 0 0 0 373 2 0 0 25 0 1 0 146831162 44961792 6376 4089446400 134512640 146736320 4293752208 18446744073709551615 135999560 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/22648/statm: 10977 6376 1032 2985 0 7519 0
Current children cumulated CPU time (s) 3.75
Current children cumulated vsize (KiB) 52608
heavy processes:

[startup+12.7436 s]
/proc/loadavg: 8.45 7.44 3.84 19/195 22664
/proc/meminfo: memFree=31153048/32951124 swapFree=67111528/67111528
[pid=22647] ppid=22645 vsize=8700 CPUtime=0
/proc/22647/stat : 22647 (run) S 22645 22647 21865 0 -1 4202496 364 0 0 0 0 0 0 0 25 0 1 0 146831162 8908800 248 4089446400 4194304 4922060 140736532750384 18446744073709551615 270338202757 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/22647/statm: 2175 248 208 178 0 68 0
[pid=22648] ppid=22647 vsize=256944 CPUtime=21.33
/proc/22648/stat : 22648 (pb_cplex) R 22647 22647 21865 0 -1 4202496 67034 0 0 0 2120 13 0 0 17 0 8 0 146831162 263110656 34041 4089446400 134512640 146736320 4293752208 18446744073709551615 136403414 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0
/proc/22648/statm: 64236 34041 1094 2985 0 60778 0
[pid=22648/tid=22658] ppid=22647 vsize=256944 CPUtime=1.65
/proc/22648/task/22658/stat : 22658 (pb_cplex) S 22647 22647 21865 0 -1 4202560 4530 0 0 0 165 0 0 0 17 0 8 0 146831967 263110656 34041 4089446400 134512640 146736320 4293752208 18446744073709551615 144580073 0 2 4096 16387 18446604471145516096 0 0 -1 1 0 0 0
[pid=22648/tid=22659] ppid=22647 vsize=256944 CPUtime=2.04
/proc/22648/task/22659/stat : 22659 (pb_cplex) R 22647 22647 21865 0 -1 4202560 2609 0 0 0 204 0 0 0 19 0 8 0 146831967 263110656 34041 4089446400 134512640 146736320 4293752208 18446744073709551615 136001974 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=22648/tid=22660] ppid=22647 vsize=256944 CPUtime=2.11
/proc/22648/task/22660/stat : 22660 (pb_cplex) R 22647 22647 21865 0 -1 4202560 3424 0 0 0 211 0 0 0 19 0 8 0 146831967 263110656 34041 4089446400 134512640 146736320 4293752208 18446744073709551615 135454079 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=22648/tid=22661] ppid=22647 vsize=256944 CPUtime=2.04
/proc/22648/task/22661/stat : 22661 (pb_cplex) S 22647 22647 21865 0 -1 4202560 3632 0 0 0 204 0 0 0 18 0 8 0 146831967 263110656 34041 4089446400 134512640 146736320 4293752208 18446744073709551615 144580073 0 2 4096 16387 18446604471145516096 0 0 -1 4 0 0 0
[pid=22648/tid=22662] ppid=22647 vsize=256944 CPUtime=2.07
/proc/22648/task/22662/stat : 22662 (pb_cplex) R 22647 22647 21865 0 -1 4202560 2371 0 0 0 206 1 0 0 19 0 8 0 146831967 263110656 34041 4089446400 134512640 146736320 4293752208 18446744073709551615 136400574 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=22648/tid=22663] ppid=22647 vsize=256944 CPUtime=1.18
/proc/22648/task/22663/stat : 22663 (pb_cplex) R 22647 22647 21865 0 -1 4202560 3808 0 0 0 118 0 0 0 18 0 8 0 146831967 263110656 34041 4089446400 134512640 146736320 4293752208 18446744073709551615 136008474 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=22648/tid=22664] ppid=22647 vsize=256944 CPUtime=1.81
/proc/22648/task/22664/stat : 22664 (pb_cplex) R 22647 22647 21865 0 -1 4202560 3921 0 0 0 181 0 0 0 18 0 8 0 146831967 263110656 34041 4089446400 134512640 146736320 4293752208 18446744073709551615 136007213 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 21.33
Current children cumulated vsize (KiB) 265644

[startup+25.5012 s]
/proc/loadavg: 8.20 7.42 3.87 10/188 22664
/proc/meminfo: memFree=31204192/32951124 swapFree=67111528/67111528
[pid=22647] ppid=22645 vsize=8700 CPUtime=0
/proc/22647/stat : 22647 (run) S 22645 22647 21865 0 -1 4202496 364 0 0 0 0 0 0 0 25 0 1 0 146831162 8908800 248 4089446400 4194304 4922060 140736532750384 18446744073709551615 270338202757 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/22647/statm: 2175 248 208 178 0 68 0
[pid=22648] ppid=22647 vsize=140184 CPUtime=46.28
/proc/22648/stat : 22648 (pb_cplex) R 22647 22647 21865 0 -1 4202496 87972 0 0 0 4606 22 0 0 25 0 1 0 146831162 143548416 20888 4089446400 134512640 146736320 4293752208 18446744073709551615 139042883 0 0 4096 16387 0 0 0 17 4 0 0 0
/proc/22648/statm: 35046 20888 1076 2985 0 31588 0
Current children cumulated CPU time (s) 46.28
Current children cumulated vsize (KiB) 148884

[startup+51.101 s]
/proc/loadavg: 8.91 7.63 4.03 19/195 22679
/proc/meminfo: memFree=31130864/32951124 swapFree=67111528/67111528
[pid=22647] ppid=22645 vsize=8700 CPUtime=0
/proc/22647/stat : 22647 (run) S 22645 22647 21865 0 -1 4202496 364 0 0 0 0 0 0 0 25 0 1 0 146831162 8908800 248 4089446400 4194304 4922060 140736532750384 18446744073709551615 270338202757 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/22647/statm: 2175 248 208 178 0 68 0
[pid=22648] ppid=22647 vsize=268408 CPUtime=76.95
/proc/22648/stat : 22648 (pb_cplex) R 22647 22647 21865 0 -1 4202496 135668 0 0 0 7662 33 0 0 25 0 8 0 146831162 274849792 39692 4089446400 134512640 146736320 4293752208 18446744073709551615 135237967 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/22648/statm: 67102 39692 1089 2985 0 63644 0
[pid=22648/tid=22673] ppid=22647 vsize=268408 CPUtime=2.19
/proc/22648/task/22673/stat : 22673 (pb_cplex) R 22647 22647 21865 0 -1 4202560 204 0 0 0 219 0 0 0 21 0 8 0 146834763 274849792 39692 4089446400 134512640 146736320 4293752208 18446744073709551615 136403109 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=22648/tid=22674] ppid=22647 vsize=268408 CPUtime=1.93
/proc/22648/task/22674/stat : 22674 (pb_cplex) R 22647 22647 21865 0 -1 4202560 2256 0 0 0 193 0 0 0 21 0 8 0 146834763 274849792 39692 4089446400 134512640 146736320 4293752208 18446744073709551615 136403117 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=22648/tid=22675] ppid=22647 vsize=268408 CPUtime=1.65
/proc/22648/task/22675/stat : 22675 (pb_cplex) R 22647 22647 21865 0 -1 4202560 147 0 0 0 165 0 0 0 20 0 8 0 146834763 274849792 39692 4089446400 134512640 146736320 4293752208 18446744073709551615 136008238 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=22648/tid=22676] ppid=22647 vsize=268408 CPUtime=2.38
/proc/22648/task/22676/stat : 22676 (pb_cplex) R 22647 22647 21865 0 -1 4202560 665 0 0 0 238 0 0 0 18 0 8 0 146834763 274849792 39692 4089446400 134512640 146736320 4293752208 18446744073709551615 139028313 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=22648/tid=22677] ppid=22647 vsize=268408 CPUtime=2.32
/proc/22648/task/22677/stat : 22677 (pb_cplex) R 22647 22647 21865 0 -1 4202560 4114 0 0 0 232 0 0 0 18 0 8 0 146834763 274849792 39692 4089446400 134512640 146736320 4293752208 18446744073709551615 136008168 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=22648/tid=22678] ppid=22647 vsize=268408 CPUtime=2.47
/proc/22648/task/22678/stat : 22678 (pb_cplex) R 22647 22647 21865 0 -1 4202560 4795 0 0 0 246 1 0 0 19 0 8 0 146834763 274849792 39692 4089446400 134512640 146736320 4293752208 18446744073709551615 136403112 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=22648/tid=22679] ppid=22647 vsize=268408 CPUtime=2.77
/proc/22648/task/22679/stat : 22679 (pb_cplex) R 22647 22647 21865 0 -1 4202560 2868 0 0 0 277 0 0 0 19 0 8 0 146834763 274849792 39692 4089446400 134512640 146736320 4293752208 18446744073709551615 139023073 0 2 4096 16387 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 76.95
Current children cumulated vsize (KiB) 277108

[startup+102.307 s]
/proc/loadavg: 12.66 8.86 4.65 11/195 22680
/proc/meminfo: memFree=31116448/32951124 swapFree=67111528/67111528
[pid=22647] ppid=22645 vsize=8700 CPUtime=0
/proc/22647/stat : 22647 (run) S 22645 22647 21865 0 -1 4202496 364 0 0 0 0 0 0 0 25 0 1 0 146831162 8908800 248 4089446400 4194304 4922060 140736532750384 18446744073709551615 270338202757 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/22647/statm: 2175 248 208 178 0 68 0
[pid=22648] ppid=22647 vsize=274132 CPUtime=169.34
/proc/22648/stat : 22648 (pb_cplex) R 22647 22647 21865 0 -1 4202496 184103 0 0 0 16883 51 0 0 21 0 8 0 146831162 280711168 42550 4089446400 134512640 146736320 4293752208 18446744073709551615 135854518 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/22648/statm: 68533 42550 1083 2985 0 65075 0
[pid=22648/tid=22673] ppid=22647 vsize=274132 CPUtime=9.8
/proc/22648/task/22673/stat : 22673 (pb_cplex) R 22647 22647 21865 0 -1 4202560 8184 0 0 0 979 1 0 0 20 0 8 0 146834763 280711168 42550 4089446400 134512640 146736320 4293752208 18446744073709551615 135454090 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=22648/tid=22674] ppid=22647 vsize=274132 CPUtime=11.18
/proc/22648/task/22674/stat : 22674 (pb_cplex) R 22647 22647 21865 0 -1 4202560 10674 0 0 0 1115 3 0 0 20 0 8 0 146834763 280711168 42550 4089446400 134512640 146736320 4293752208 18446744073709551615 136382629 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=22648/tid=22675] ppid=22647 vsize=274132 CPUtime=16.58
/proc/22648/task/22675/stat : 22675 (pb_cplex) R 22647 22647 21865 0 -1 4202560 153 0 0 0 1656 2 0 0 25 0 8 0 146834763 280711168 42550 4089446400 134512640 146736320 4293752208 18446744073709551615 136388871 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=22648/tid=22676] ppid=22647 vsize=274132 CPUtime=15.43
/proc/22648/task/22676/stat : 22676 (pb_cplex) R 22647 22647 21865 0 -1 4202560 7394 0 0 0 1540 3 0 0 20 0 8 0 146834763 280711168 42550 4089446400 134512640 146736320 4293752208 18446744073709551615 136008230 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=22648/tid=22677] ppid=22647 vsize=274132 CPUtime=10.93
/proc/22648/task/22677/stat : 22677 (pb_cplex) R 22647 22647 21865 0 -1 4202560 7338 0 0 0 1092 1 0 0 20 0 8 0 146834763 280711168 42550 4089446400 134512640 146736320 4293752208 18446744073709551615 136007998 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=22648/tid=22678] ppid=22647 vsize=274132 CPUtime=14.14
/proc/22648/task/22678/stat : 22678 (pb_cplex) R 22647 22647 21865 0 -1 4202560 10124 0 0 0 1410 4 0 0 20 0 8 0 146834763 280711168 42550 4089446400 134512640 146736320 4293752208 18446744073709551615 139023058 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=22648/tid=22679] ppid=22647 vsize=274132 CPUtime=14.06
/proc/22648/task/22679/stat : 22679 (pb_cplex) R 22647 22647 21865 0 -1 4202560 7895 0 0 0 1404 2 0 0 20 0 8 0 146834763 280711168 42550 4089446400 134512640 146736320 4293752208 18446744073709551615 136008056 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 169.34
Current children cumulated vsize (KiB) 282832

[startup+162.301 s]
/proc/loadavg: 12.00 9.39 5.10 9/195 22696
/proc/meminfo: memFree=31114396/32951124 swapFree=67111528/67111528
[pid=22647] ppid=22645 vsize=8700 CPUtime=0
/proc/22647/stat : 22647 (run) S 22645 22647 21865 0 -1 4202496 364 0 0 0 0 0 0 0 25 0 1 0 146831162 8908800 248 4089446400 4194304 4922060 140736532750384 18446744073709551615 270338202757 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/22647/statm: 2175 248 208 178 0 68 0
[pid=22648] ppid=22647 vsize=288080 CPUtime=308.42
/proc/22648/stat : 22648 (pb_cplex) S 22647 22647 21865 0 -1 4202496 229038 0 0 0 30770 72 0 0 25 0 8 0 146831162 294993920 45340 4089446400 134512640 146736320 4293752208 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0
/proc/22648/statm: 72020 45340 1098 2985 0 68562 0
[pid=22648/tid=22673] ppid=22647 vsize=288080 CPUtime=25.95
/proc/22648/task/22673/stat : 22673 (pb_cplex) R 22647 22647 21865 0 -1 4202560 13678 0 0 0 2591 4 0 0 23 0 8 0 146834763 294993920 45340 4089446400 134512640 146736320 4293752208 18446744073709551615 136008157 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=22648/tid=22674] ppid=22647 vsize=288080 CPUtime=26.86
/proc/22648/task/22674/stat : 22674 (pb_cplex) S 22647 22647 21865 0 -1 4202560 18199 0 0 0 2680 6 0 0 25 0 8 0 146834763 294993920 45340 4089446400 134512640 146736320 4293752208 18446744073709551615 144580073 0 2 4096 16387 18446604471145516096 0 0 -1 2 0 0 0
[pid=22648/tid=22675] ppid=22647 vsize=288080 CPUtime=35.73
/proc/22648/task/22675/stat : 22675 (pb_cplex) R 22647 22647 21865 0 -1 4202560 153 0 0 0 3570 3 0 0 25 0 8 0 146834763 294993920 45340 4089446400 134512640 146736320 4293752208 18446744073709551615 139044550 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=22648/tid=22676] ppid=22647 vsize=288080 CPUtime=34.17
/proc/22648/task/22676/stat : 22676 (pb_cplex) R 22647 22647 21865 0 -1 4202560 9219 0 0 0 3412 5 0 0 25 0 8 0 146834763 294993920 45340 4089446400 134512640 146736320 4293752208 18446744073709551615 136384861 0 2 4096 16387 0 0 0 -1 5 0 0 0

################
# More data... #
################

/proc/22648/statm: 78644 51845 1083 2985 0 75186 0
[pid=22648/tid=22673] ppid=22647 vsize=314576 CPUtime=129.74
/proc/22648/task/22673/stat : 22673 (pb_cplex) R 22647 22647 21865 0 -1 4202560 91974 0 0 0 12949 25 0 0 20 0 8 0 146834763 322125824 51845 4089446400 134512640 146736320 4293752208 18446744073709551615 136400469 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=22648/tid=22674] ppid=22647 vsize=314576 CPUtime=128
/proc/22648/task/22674/stat : 22674 (pb_cplex) R 22647 22647 21865 0 -1 4202560 58031 0 0 0 12777 23 0 0 16 0 8 0 146834763 322125824 51845 4089446400 134512640 146736320 4293752208 18446744073709551615 136008167 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=22648/tid=22675] ppid=22647 vsize=314576 CPUtime=139.72
/proc/22648/task/22675/stat : 22675 (pb_cplex) R 22647 22647 21865 0 -1 4202560 37179 0 0 0 13955 17 0 0 21 0 8 0 146834763 322125824 51845 4089446400 134512640 146736320 4293752208 18446744073709551615 136403109 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=22648/tid=22676] ppid=22647 vsize=314576 CPUtime=138.42
/proc/22648/task/22676/stat : 22676 (pb_cplex) R 22647 22647 21865 0 -1 4202560 66718 0 0 0 13815 27 0 0 25 0 8 0 146834763 322125824 51845 4089446400 134512640 146736320 4293752208 18446744073709551615 136008163 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=22648/tid=22677] ppid=22647 vsize=314576 CPUtime=131.63
/proc/22648/task/22677/stat : 22677 (pb_cplex) R 22647 22647 21865 0 -1 4202560 73626 0 0 0 13141 22 0 0 17 0 8 0 146834763 322125824 51845 4089446400 134512640 146736320 4293752208 18446744073709551615 136003546 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=22648/tid=22678] ppid=22647 vsize=314576 CPUtime=137.11
/proc/22648/task/22678/stat : 22678 (pb_cplex) R 22647 22647 21865 0 -1 4202560 70617 0 0 0 13688 23 0 0 17 0 8 0 146834763 322125824 51845 4089446400 134512640 146736320 4293752208 18446744073709551615 136000480 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=22648/tid=22679] ppid=22647 vsize=314576 CPUtime=133.23
/proc/22648/task/22679/stat : 22679 (pb_cplex) R 22647 22647 21865 0 -1 4202560 70671 0 0 0 13302 21 0 0 17 0 8 0 146834763 322125824 51845 4089446400 134512640 146736320 4293752208 18446744073709551615 135459442 0 2 4096 16387 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 1133.56
Current children cumulated vsize (KiB) 323276

[startup+582.301 s]
/proc/loadavg: 14.92 13.13 8.39 14/195 22872
/proc/meminfo: memFree=31067628/32951124 swapFree=67111528/67111528
[pid=22647] ppid=22645 vsize=8700 CPUtime=0
/proc/22647/stat : 22647 (run) S 22645 22647 21865 0 -1 4202496 364 0 0 0 0 0 0 0 25 0 1 0 146831162 8908800 248 4089446400 4194304 4922060 140736532750384 18446744073709551615 270338202757 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/22647/statm: 2175 248 208 178 0 68 0
[pid=22648] ppid=22647 vsize=326160 CPUtime=1274.38
/proc/22648/stat : 22648 (pb_cplex) R 22647 22647 21865 0 -1 4202496 671958 0 0 0 127210 228 0 0 20 0 8 0 146831162 333987840 54161 4089446400 134512640 146736320 4293752208 18446744073709551615 136007270 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/22648/statm: 81540 54161 1083 2985 0 78082 0
[pid=22648/tid=22673] ppid=22647 vsize=326160 CPUtime=149.15
/proc/22648/task/22673/stat : 22673 (pb_cplex) R 22647 22647 21865 0 -1 4202560 98132 0 0 0 14887 28 0 0 22 0 8 0 146834763 333987840 54161 4089446400 134512640 146736320 4293752208 18446744073709551615 139046505 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=22648/tid=22674] ppid=22647 vsize=326160 CPUtime=143.61
/proc/22648/task/22674/stat : 22674 (pb_cplex) R 22647 22647 21865 0 -1 4202560 64560 0 0 0 14336 25 0 0 21 0 8 0 146834763 333987840 54161 4089446400 134512640 146736320 4293752208 18446744073709551615 136008168 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=22648/tid=22675] ppid=22647 vsize=326160 CPUtime=157.06
/proc/22648/task/22675/stat : 22675 (pb_cplex) R 22647 22647 21865 0 -1 4202560 48245 0 0 0 15686 20 0 0 20 0 8 0 146834763 333987840 54161 4089446400 134512640 146736320 4293752208 18446744073709551615 139042885 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=22648/tid=22676] ppid=22647 vsize=326160 CPUtime=156.17
/proc/22648/task/22676/stat : 22676 (pb_cplex) R 22647 22647 21865 0 -1 4202560 71430 0 0 0 15587 30 0 0 21 0 8 0 146834763 333987840 54161 4089446400 134512640 146736320 4293752208 18446744073709551615 136008111 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=22648/tid=22677] ppid=22647 vsize=326160 CPUtime=149.13
/proc/22648/task/22677/stat : 22677 (pb_cplex) R 22647 22647 21865 0 -1 4202560 81644 0 0 0 14888 25 0 0 20 0 8 0 146834763 333987840 54161 4089446400 134512640 146736320 4293752208 18446744073709551615 136008020 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=22648/tid=22678] ppid=22647 vsize=326160 CPUtime=153.52
/proc/22648/task/22678/stat : 22678 (pb_cplex) R 22647 22647 21865 0 -1 4202560 72809 0 0 0 15327 25 0 0 19 0 8 0 146834763 333987840 54161 4089446400 134512640 146736320 4293752208 18446744073709551615 136007327 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=22648/tid=22679] ppid=22647 vsize=326160 CPUtime=152.81
/proc/22648/task/22679/stat : 22679 (pb_cplex) R 22647 22647 21865 0 -1 4202560 80235 0 0 0 15257 24 0 0 25 0 8 0 146834763 333987840 54161 4089446400 134512640 146736320 4293752208 18446744073709551615 136008450 0 2 4096 16387 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1274.38
Current children cumulated vsize (KiB) 334860

[startup+642.354 s]
/proc/loadavg: 14.92 13.43 8.78 19/195 22874
/proc/meminfo: memFree=31069772/32951124 swapFree=67111528/67111528
[pid=22647] ppid=22645 vsize=8700 CPUtime=0
/proc/22647/stat : 22647 (run) S 22645 22647 21865 0 -1 4202496 364 0 0 0 0 0 0 0 25 0 1 0 146831162 8908800 248 4089446400 4194304 4922060 140736532750384 18446744073709551615 270338202757 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/22647/statm: 2175 248 208 178 0 68 0
[pid=22648] ppid=22647 vsize=319092 CPUtime=1378.22
/proc/22648/stat : 22648 (pb_cplex) R 22647 22647 21865 0 -1 4202496 718893 0 0 0 137574 248 0 0 19 0 8 0 146831162 326750208 52834 4089446400 134512640 146736320 4293752208 18446744073709551615 136008168 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0
/proc/22648/statm: 79773 52834 1083 2985 0 76315 0
[pid=22648/tid=22673] ppid=22647 vsize=319092 CPUtime=160.74
/proc/22648/task/22673/stat : 22673 (pb_cplex) R 22647 22647 21865 0 -1 4202560 102215 0 0 0 16044 30 0 0 25 0 8 0 146834763 326750208 52834 4089446400 134512640 146736320 4293752208 18446744073709551615 136403114 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=22648/tid=22674] ppid=22647 vsize=319092 CPUtime=156.3
/proc/22648/task/22674/stat : 22674 (pb_cplex) R 22647 22647 21865 0 -1 4202560 73426 0 0 0 15603 27 0 0 18 0 8 0 146834763 326750208 52834 4089446400 134512640 146736320 4293752208 18446744073709551615 136000506 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=22648/tid=22675] ppid=22647 vsize=319092 CPUtime=169.91
/proc/22648/task/22675/stat : 22675 (pb_cplex) R 22647 22647 21865 0 -1 4202560 57362 0 0 0 16968 23 0 0 17 0 8 0 146834763 326750208 52834 4089446400 134512640 146736320 4293752208 18446744073709551615 136400570 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=22648/tid=22676] ppid=22647 vsize=319092 CPUtime=169.06
/proc/22648/task/22676/stat : 22676 (pb_cplex) R 22647 22647 21865 0 -1 4202560 71431 0 0 0 16875 31 0 0 25 0 8 0 146834763 326750208 52834 4089446400 134512640 146736320 4293752208 18446744073709551615 136008050 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=22648/tid=22677] ppid=22647 vsize=319092 CPUtime=164.94
/proc/22648/task/22677/stat : 22677 (pb_cplex) R 22647 22647 21865 0 -1 4202560 88583 0 0 0 16465 29 0 0 21 0 8 0 146834763 326750208 52834 4089446400 134512640 146736320 4293752208 18446744073709551615 136388081 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=22648/tid=22678] ppid=22647 vsize=319092 CPUtime=166.13
/proc/22648/task/22678/stat : 22678 (pb_cplex) R 22647 22647 21865 0 -1 4202560 79681 0 0 0 16586 27 0 0 19 0 8 0 146834763 326750208 52834 4089446400 134512640 146736320 4293752208 18446744073709551615 136008245 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=22648/tid=22679] ppid=22647 vsize=319092 CPUtime=165.51
/proc/22648/task/22679/stat : 22679 (pb_cplex) R 22647 22647 21865 0 -1 4202560 86750 0 0 0 16525 26 0 0 18 0 8 0 146834763 326750208 52834 4089446400 134512640 146736320 4293752208 18446744073709551615 139042916 0 2 4096 16387 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1378.22
Current children cumulated vsize (KiB) 327792

[startup+702.3 s]
/proc/loadavg: 11.05 12.65 8.81 9/179 22904
/proc/meminfo: memFree=31147928/32951124 swapFree=67111528/67111528
[pid=22647] ppid=22645 vsize=8700 CPUtime=0
/proc/22647/stat : 22647 (run) S 22645 22647 21865 0 -1 4202496 364 0 0 0 0 0 0 0 25 0 1 0 146831162 8908800 248 4089446400 4194304 4922060 140736532750384 18446744073709551615 270338202757 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/22647/statm: 2175 248 208 178 0 68 0
[pid=22648] ppid=22647 vsize=321552 CPUtime=1687.3
/proc/22648/stat : 22648 (pb_cplex) R 22647 22647 21865 0 -1 4202496 844856 0 0 0 168441 289 0 0 25 0 8 0 146831162 329269248 53282 4089446400 134512640 146736320 4293752208 18446744073709551615 136000480 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0
/proc/22648/statm: 80388 53282 1083 2985 0 76930 0
[pid=22648/tid=22673] ppid=22647 vsize=321552 CPUtime=199.01
/proc/22648/task/22673/stat : 22673 (pb_cplex) R 22647 22647 21865 0 -1 4202560 112463 0 0 0 19867 34 0 0 25 0 8 0 146834763 329269248 53282 4089446400 134512640 146736320 4293752208 18446744073709551615 139044505 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=22648/tid=22674] ppid=22647 vsize=321552 CPUtime=194.79
/proc/22648/task/22674/stat : 22674 (pb_cplex) R 22647 22647 21865 0 -1 4202560 87354 0 0 0 19448 31 0 0 25 0 8 0 146834763 329269248 53282 4089446400 134512640 146736320 4293752208 18446744073709551615 136461822 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=22648/tid=22675] ppid=22647 vsize=321552 CPUtime=208.13
/proc/22648/task/22675/stat : 22675 (pb_cplex) R 22647 22647 21865 0 -1 4202560 81079 0 0 0 20783 30 0 0 25 0 8 0 146834763 329269248 53282 4089446400 134512640 146736320 4293752208 18446744073709551615 136008050 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=22648/tid=22676] ppid=22647 vsize=321552 CPUtime=211
/proc/22648/task/22676/stat : 22676 (pb_cplex) R 22647 22647 21865 0 -1 4202560 85676 0 0 0 21064 36 0 0 25 0 8 0 146834763 329269248 53282 4089446400 134512640 146736320 4293752208 18446744073709551615 136388081 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=22648/tid=22677] ppid=22647 vsize=321552 CPUtime=201.62
/proc/22648/task/22677/stat : 22677 (pb_cplex) R 22647 22647 21865 0 -1 4202560 108138 0 0 0 20129 33 0 0 25 0 8 0 146834763 329269248 53282 4089446400 134512640 146736320 4293752208 18446744073709551615 136403112 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=22648/tid=22678] ppid=22647 vsize=321552 CPUtime=205.31
/proc/22648/task/22678/stat : 22678 (pb_cplex) R 22647 22647 21865 0 -1 4202560 97286 0 0 0 20499 32 0 0 25 0 8 0 146834763 329269248 53282 4089446400 134512640 146736320 4293752208 18446744073709551615 136008065 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=22648/tid=22679] ppid=22647 vsize=321552 CPUtime=201.58
/proc/22648/task/22679/stat : 22679 (pb_cplex) R 22647 22647 21865 0 -1 4202560 100781 0 0 0 20126 32 0 0 25 0 8 0 146834763 329269248 53282 4089446400 134512640 146736320 4293752208 18446744073709551615 136007990 0 2 4096 16387 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1687.3
Current children cumulated vsize (KiB) 330252



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+720.201 s]
/proc/loadavg: 10.15 12.37 8.78 8/179 22905
/proc/meminfo: memFree=31141256/32951124 swapFree=67111528/67111528
[pid=22647] ppid=22645 vsize=8700 CPUtime=0
/proc/22647/stat : 22647 (run) S 22645 22647 21865 0 -1 4202496 364 0 0 0 0 0 0 0 25 0 1 0 146831162 8908800 248 4089446400 4194304 4922060 140736532750384 18446744073709551615 270338202757 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/22647/statm: 2175 248 208 178 0 68 0
[pid=22648] ppid=22647 vsize=333120 CPUtime=1800.37
/proc/22648/stat : 22648 (pb_cplex) R 22647 22647 21865 0 -1 4202496 901530 0 0 0 179732 305 0 0 25 0 8 0 146831162 341114880 55614 4089446400 134512640 146736320 4293752208 18446744073709551615 135999699 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0
/proc/22648/statm: 83280 55614 1100 2985 0 79822 0
[pid=22648/tid=22673] ppid=22647 vsize=333120 CPUtime=213.51
/proc/22648/task/22673/stat : 22673 (pb_cplex) R 22647 22647 21865 0 -1 4202560 122063 0 0 0 21315 36 0 0 25 0 8 0 146834763 341114880 55614 4089446400 134512640 146736320 4293752208 18446744073709551615 136475778 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=22648/tid=22674] ppid=22647 vsize=333120 CPUtime=208.51
/proc/22648/task/22674/stat : 22674 (pb_cplex) R 22647 22647 21865 0 -1 4202560 96430 0 0 0 20818 33 0 0 22 0 8 0 146834763 341114880 55614 4089446400 134512640 146736320 4293752208 18446744073709551615 136403109 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=22648/tid=22675] ppid=22647 vsize=333120 CPUtime=221.92
/proc/22648/task/22675/stat : 22675 (pb_cplex) R 22647 22647 21865 0 -1 4202560 89417 0 0 0 22160 32 0 0 19 0 8 0 146834763 341114880 55614 4089446400 134512640 146736320 4293752208 18446744073709551615 135850315 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=22648/tid=22676] ppid=22647 vsize=333120 CPUtime=224.39
/proc/22648/task/22676/stat : 22676 (pb_cplex) R 22647 22647 21865 0 -1 4202560 94033 0 0 0 22400 39 0 0 25 0 8 0 146834763 341114880 55614 4089446400 134512640 146736320 4293752208 18446744073709551615 136006076 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=22648/tid=22677] ppid=22647 vsize=333120 CPUtime=217.11
/proc/22648/task/22677/stat : 22677 (pb_cplex) R 22647 22647 21865 0 -1 4202560 113938 0 0 0 21676 35 0 0 25 0 8 0 146834763 341114880 55614 4089446400 134512640 146736320 4293752208 18446744073709551615 136008050 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=22648/tid=22678] ppid=22647 vsize=333120 CPUtime=220.43
/proc/22648/task/22678/stat : 22678 (pb_cplex) S 22647 22647 21865 0 -1 4202560 102284 0 0 0 22010 33 0 0 24 0 8 0 146834763 341114880 55614 4089446400 134512640 146736320 4293752208 18446744073709551615 144580073 0 2 4096 16387 18446604471145516096 0 0 -1 3 0 0 0
[pid=22648/tid=22679] ppid=22647 vsize=333120 CPUtime=214.29
/proc/22648/task/22679/stat : 22679 (pb_cplex) R 22647 22647 21865 0 -1 4202560 102895 0 0 0 21396 33 0 0 25 0 8 0 146834763 341114880 55614 4089446400 134512640 146736320 4293752208 18446744073709551615 136007605 0 2 4096 16387 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1800.37
Current children cumulated vsize (KiB) 341820

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+720.201 s]
/proc/loadavg: 10.15 12.37 8.78 8/179 22905
/proc/meminfo: memFree=31141256/32951124 swapFree=67111528/67111528
[pid=22647] ppid=22645 vsize=8700 CPUtime=0
/proc/22647/stat : 22647 (run) S 22645 22647 21865 0 -1 4202496 364 0 0 0 0 0 0 0 25 0 1 0 146831162 8908800 248 4089446400 4194304 4922060 140736532750384 18446744073709551615 270338202757 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/22647/statm: 2175 248 208 178 0 68 0
[pid=22648] ppid=22647 vsize=333120 CPUtime=1800.37
/proc/22648/stat : 22648 (pb_cplex) R 22647 22647 21865 0 -1 4202496 901530 0 0 0 179732 305 0 0 25 0 8 0 146831162 341114880 55614 4089446400 134512640 146736320 4293752208 18446744073709551615 135999699 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0
/proc/22648/statm: 83280 55614 1100 2985 0 79822 0
[pid=22648/tid=22673] ppid=22647 vsize=333120 CPUtime=213.51
/proc/22648/task/22673/stat : 22673 (pb_cplex) R 22647 22647 21865 0 -1 4202560 122063 0 0 0 21315 36 0 0 25 0 8 0 146834763 341114880 55614 4089446400 134512640 146736320 4293752208 18446744073709551615 136475778 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=22648/tid=22674] ppid=22647 vsize=333120 CPUtime=208.51
/proc/22648/task/22674/stat : 22674 (pb_cplex) R 22647 22647 21865 0 -1 4202560 96430 0 0 0 20818 33 0 0 22 0 8 0 146834763 341114880 55614 4089446400 134512640 146736320 4293752208 18446744073709551615 136403109 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=22648/tid=22675] ppid=22647 vsize=333120 CPUtime=221.92
/proc/22648/task/22675/stat : 22675 (pb_cplex) R 22647 22647 21865 0 -1 4202560 89417 0 0 0 22160 32 0 0 19 0 8 0 146834763 341114880 55614 4089446400 134512640 146736320 4293752208 18446744073709551615 135850315 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=22648/tid=22676] ppid=22647 vsize=333120 CPUtime=224.39
/proc/22648/task/22676/stat : 22676 (pb_cplex) R 22647 22647 21865 0 -1 4202560 94033 0 0 0 22400 39 0 0 25 0 8 0 146834763 341114880 55614 4089446400 134512640 146736320 4293752208 18446744073709551615 136006076 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=22648/tid=22677] ppid=22647 vsize=333120 CPUtime=217.11
/proc/22648/task/22677/stat : 22677 (pb_cplex) R 22647 22647 21865 0 -1 4202560 113938 0 0 0 21676 35 0 0 25 0 8 0 146834763 341114880 55614 4089446400 134512640 146736320 4293752208 18446744073709551615 136008050 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=22648/tid=22678] ppid=22647 vsize=333120 CPUtime=220.43
/proc/22648/task/22678/stat : 22678 (pb_cplex) S 22647 22647 21865 0 -1 4202560 102284 0 0 0 22010 33 0 0 24 0 8 0 146834763 341114880 55614 4089446400 134512640 146736320 4293752208 18446744073709551615 144580073 0 2 4096 16387 18446604471145516096 0 0 -1 3 0 0 0
[pid=22648/tid=22679] ppid=22647 vsize=333120 CPUtime=214.29
/proc/22648/task/22679/stat : 22679 (pb_cplex) R 22647 22647 21865 0 -1 4202560 102895 0 0 0 21396 33 0 0 25 0 8 0 146834763 341114880 55614 4089446400 134512640 146736320 4293752208 18446744073709551615 136007605 0 2 4096 16387 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1800.37
Current children cumulated vsize (KiB) 341820

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 22647 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=0
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=1999
# CPU time returned by wait4() is 0.001999
# while last known CPU time is 1800.37
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 720.217
CPU time (s): 1800.37
CPU user time (s): 1797.32
CPU system time (s): 3.05
CPU usage (%): 249.976
Max. virtual memory (cumulated for all children) (KiB): 353564

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0
system time used= 0.001999
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 364
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2
involuntary context switches= 2

runsolver used 1.35979 second user time and 2.83657 second system time

The end

Launcher Data 

Begin job on node143 at 2010-07-02 09:22:40
IDJOB=2697508
IDBENCH=79053
IDSOLVER=1210
FILE ID=node143/2697508-1278055359
PBS_JOBID= 11198895
Free space on /tmp= 74188 MiB

SOLVER NAME= pb_cplex 2010-07-01
BENCH NAME= PB10/normalized-PB10/PARTIAL-SMALLINT-LIN/PB06/web/uclid_pb_benchmarks/normalized-ooo.tag8.ucl--soft-66-100-0.wbo
COMMAND LINE= run BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-2697508-1278055359/watcher-2697508-1278055359 -o /tmp/evaluation-result-2697508-1278055359/solver-2697508-1278055359 -C 1800 -W 2000 -M 3800  run HOME/instance-2697508-1278055359.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 3800 MiB
MAX NB THREAD= 0

MD5SUM BENCH= fbaddd9625623c5e0a4f0ddddf2cf090
RANDOM SEED=1570903224

node143.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5285.38
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      31286932 kB
Buffers:        225124 kB
Cached:        1173304 kB
SwapCached:          0 kB
Active:         615208 kB
Inactive:       897688 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      31286932 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           17456 kB
Writeback:           0 kB
AnonPages:      114928 kB
Mapped:          19172 kB
Slab:            83776 kB
PageTables:       5696 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   336884 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 74188 MiB
End job on node143 at 2010-07-02 09:34:42