Trace number 3703663

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
pwbo 2.0SAT (TO)76 1800.12 900.314

General information on the benchmark

Namenormalized-PB06/OPT-SMALLINT/submitted-PB06/manquiho/
bounded_golomb_rulers/normalized-bogr_11.opb
MD5SUMc19ad978a977652a94fc03b8ce63204a
Bench CategoryOPT-SMALLINT (optimisation, small integers)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark72
Best CPU time to get the best result obtained on this benchmark1800.01
Has Objective FunctionYES
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function 74
Optimality of the best value was proved NO
Number of variables1288
Total number of constraints2431
Number of constraints which are clauses1
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints2430
Minimum length of a constraint1
Maximum length of a constraint29
Number of terms in the objective function 8
Biggest coefficient in the objective function 64
Number of bits for the biggest coefficient in the objective function 7
Sum of the numbers in the objective function 182
Number of bits of the sum of numbers in the objective function 8
Biggest number in a constraint 175
Number of bits of the biggest number in a constraint 8
Biggest sum of numbers in a constraint 802
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.00/0.00	c ==============================[ Solver Version ]===============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  pwbo:                    2.0 beta - March, 2012                            |
0.00/0.00	c |                                                                             |
0.00/0.00	c Parsing opb file format. File HOME/instance-3703663-1338095060.opb.
0.00/0.00	c Instance file HOME/instance-3703663-1338095060.opb
0.00/0.00	c File size is 481988 bytes.
0.00/0.04	c Cost Offset: 0
0.00/0.04	c Highest Coefficient sum: 802
0.00/0.04	c ============================[ Problem Statistics ]=============================
0.00/0.04	c |                                                                             |
0.00/0.04	c |  Parsing time:          0.05         s                                      |
0.00/0.04	c |  Number Variables:      1288                                                |
0.00/0.04	c |  Number Clauses:        1                                                   |
0.00/0.04	c |  Number Cardinality:    0                                                   |
0.00/0.04	c |  Number PB Constraints: 2430                                                |
0.00/0.04	c |                                                                             |
0.00/0.05	c |  Problem Type:          Optimization                                        |
0.00/0.05	c |  Objective Function:    Cardinality Constraint                              |
0.00/0.05	c |                                                                             |
0.00/0.05	c ===============================================================================
1800.12/900.31	o 119
1800.12/900.31	o 116
1800.12/900.31	o 114
1800.12/900.31	o 109
1800.12/900.31	o 108
1800.12/900.31	o 102
1800.12/900.31	o 96
1800.12/900.31	o 95
1800.12/900.31	o 93
1800.12/900.31	o 92
1800.12/900.31	o 87
1800.12/900.31	o 85
1800.12/900.31	o 83
1800.12/900.31	o 77
1800.12/900.31	o 76
1800.12/900.31	c ===============================================================================
1800.12/900.31	c Wall Clock Time: 900.31 s
1800.12/900.31	c Best Solution: 76
1800.12/900.31	s SATISFIABLE
1800.12/900.31	v x1 -x2 x3 -x4 x5 -x6 -x7 x1288 -x64 -x65 -x66 x67 x68 -x69 -x70 x71 -x72 x73 -x74 x75 -x76 -x77 x1287 -x57 -x58 -x59 x60 x61 -x62 -x63 -x1286 x1285 x1284 x1283 -x1282 -x50 -x51 x52 -x53 x54 -x55 -x56 -x1281 -x1280 -x1279 -x1278 -x1277 -x1276 x1275 -x1274 -x1273 x1272 -x1271 x1270 x1269 -x1268 -x43 x44 x45 x46 -x47 -x48 -x49 -x1267 -x1266 -x1265 -x1264 -x1263 -x1262 -x1261 -x1260 -x1259 x1258 -x1257 -x1256 -x1255 -x1254 x1253 -x1252 -x1251 x1250 x1249 -x1248 -x1247 x1246 -x1245 x1244 x1243 -x1242 x1241 x1240 x1239 -x1238 -x36 x37 -x38 x39 -x40 -x41 -x42 -x1237 -x1236 -x1235 -x1234 x1233 -x1232 -x1231 -x1230 -x1229 -x1228 -x1227 -x1226 x1225 -x1224 -x1223 -x1222 -x1221 -x1220 -x1219 -x1218 x1217 -x1216 -x1215 x1214 x1213 -x1212 -x1211 -x1210 -x1209 x1208 -x1207 -x1206 x1205 x1204 -x1203 -x1202 x1201 x1200 x1199 x1198 x1197 x1196 -x1195 x1194 x1193 -x1192 x1191 x1190 x1189 x1188 x1187 x1186 x1185 x1184 x1183 x29 -x30 -x31 -x32 -x33 -x34 -x35 -x1182 -x1181 -x1180 -x1179 -x1178 -x1177 -x1176 -x1175 -x1174 -x1173 -x1172 -x1171 -x1170 -x1169 -x1168 x1167 -x1166 -x1165 -x1164 -x1163 -x1162 -x1161 -x1160 -x1159 -x1158 -x1157 x1156 -x1155 -x1154 x1153 x1152 -x1151 -x1150 -x1149 -x1148 -x1147 -x1146 -x1145 x1144 -x1143 -x1142 -x1141 x1140 x1139 -x1138 -x1137 x1136 x1135 x1134 -x1133 -x1132 -x1131 -x1130 x1129 -x1128 -x1127 x1126 x1125 -x1124 -x1123 x1122 x1121 x1120 x1119 -x1118 x1117 x1116 x1115 x1114 -x1113 -x1112 x1111 -x1110 x1109 x1108 -x1107 x1106 x1105 x1104 -x1103 x1102 x1101 x1100 x1099 -x1098 x1097 x1096 x1095 x1094 x1093 -x1092 -x22 -x23 -x24 -x25 -x26 -x27 -x28 -x1091 -x1090 -x1089 -x1088 -x1087 -x1086 x1085 -x1084 -x1083 -x1082 -x1081 -x1080 -x1079 -x1078 -x1077 -x1076 -x1075 -x1074 x1073 x1072 -x1071 -x1070 -x1069 -x1068 -x1067 -x1066 -x1065 -x1064 -x1063 -x1062 -x1061 -x1060 x1059 -x1058 -x1057 x1056 x1055 x1054 -x1053 -x1052 -x1051 -x1050 -x1049 -x1048 -x1047 -x1046 -x1045 -x1044 x1043 -x1042 -x1041 -x1040 x1039 x1038 x1037 -x1036 x1035 x1034 x1033 x1032 -x1031 -x1030 -x1029 -x1028 -x1027 -x1026 x1025 -x1024 -x1023 -x1022 x1021 x1020 x1019 -x1018 -x1017 x1016 x1015 x1014 x1013 -x1012 x1011 x1010 x1009 x1008 x1007 -x1006 -x1005 -x1004 x1003 -x1002 -x1001 x1000 x999 x998 -x997 x996 x995 x994 x993 -x992 x991 x990 x989 x988 x987 -x986 x985 x984 x983 x982 x981 x980 x979 x978 x977 x976 x975 x974 x973 x972 x971 x970 x969 x968 x967 x966 x965 x964 x963 x962 x961 x960 x959 x958 x957 x956 x955 x954 x953 x952 x8 -x9 -x10 -x11 -x12 -x13 -x14 -x951 -x950 -x949 -x948 -x947 -x946 x945 x944 -x943 -x942 -x941 -x940 -x939 -x938 -x937 -x936 -x935 -x934 -x933 x932 x931 x930 x929 -x928 -x927 -x926 -x925 -x924 -x923 -x922 -x921 -x920 -x919 -x918 -x917 x916 -x915 -x914 x913 x912 x911 x910 x909 x908 -x907 -x906 -x905 -x904 -x903 -x902 -x901 -x900 -x899 -x898 x897 -x896 -x895 -x894 x893 x892 x891 -x890 x889 x888 x887 x886 x885 x884 x883 x882 -x881 -x880 -x879 -x878 -x877 -x876 x875 -x874 -x873 -x872 x871 x870 x869 -x868 -x867 x866 x865 x864 x863 -x862 x861 x860 x859 x858 x857 x856 x855 x854 x853 x852 -x851 -x850 -x849 x848 -x847 -x846 x845 x844 x843 -x842 x841 x840 x839 x838 -x837 x836 x835 x834 x833 x832 -x831 x830 x829 x828 x827 x826 x825 x824 x823 x822 x821 x820 x819 x818 x817 x816 x815 x814 x813 x812 x811 x810 x809 x808 x807 x806 x805 x804 x803 x802 x801 x800 x799 x798 x797 x796 x795 x794 x793 x792 x791 x790 x789 x788 x787 x786 x785 x784 x783 x782 x781 x780 x779 x778 x777 x776 x775 x774 x773 x772 x771 x770 x769 x768 x767 x766 x765 x764 x763 x762 x761 x760 x759 x758 x757 x756 x755 x754 x753 x752 x751 x750 x749 x748 -x15 -x16 -x17 -x18 -x19 -x20 -x21 -x747 -x746 -x745 -x744 -x743 -x742 x741 x740 x739 -x738 -x737 -x736 -x735 -x734 -x733 -x732 -x731 -x730 -x729 -x728 x727 -x726 x725 x724 x723 x722 -x721 -x720 -x719 -x718 -x717 -x716 -x715 -x714 -x713 -x712 -x711 -x710 x709 -x708 -x707 x706 x705 x704 x703 x702 x701 x700 x699 x698 -x697 -x696 -x695 -x694 -x693 -x692 -x691 -x690 -x689 -x688 x687 -x686 -x685 -x684 x683 x682 -x681 -x680 x679 x678 x677 x676 x675 x674 x673 x672 x671 x670 x669 x668 -x667 -x666 -x665 -x664 -x663 -x662 x661 -x660 -x659 -x658 x657 x656 x655 -x654 -x653 x652 x651 x650 x649 -x648 x647 x646 x645 x644 x643 x642 x641 x640 x639 x638 x637 x636 x635 x634 x633 -x632 -x631 -x630 x629 -x628 -x627 x626 x625 -x624 -x623 x622 x621 x620 x619 -x618 x617 x616 x615 x614 x613 -x612 x611 x610 x609 x608 x607 x606 x605 x604 x603 x602 x601 x600 x599 x598 x597 x596 x595 x594 x593 x592 -x591 x590 x589 x588 x587 x586 x585 x584 x583 x582 x581 x580 x579 x578 x577 x576 x575 x574 x573 x572 x571 x570 x569 x568 x567 x566 x565 x564 x563 x562 x561 x560 x559 x558 x557 x556 x555 x554 x553 x552 x551 x550 x549 x548 x547 x546 x545 x544 x543 x542 x541 x540 x539 x538 x537 x536 x535 x534 x533 x532 x531 x530 x529 x528 x527 x526 x525 x524 x523 x522 x521 x520 x519 x518 x517 x516 x515 x514 x513 x512 x511 x510 x509 x508 x507 x506 x505 x504 x503 x502 x501 x500 x499 x498 x497 x496 x495 x494 x493 x492 x491 x490 x489 x488 x487 x486 x485 x484 x483 x482 x481 x480 x479 x478 x477 x476 x475 x474 x473 x472 x471 x470 x469 x468 x467 x466 x465 x464 -x463 -x462 -x461 -x460 -x459 -x458 -x457 x456 x455 x454 x453 -x452 -x451 -x450 -x449 -x448 -x447 -x446 -x445 -x444 -x443 -x442 x441 -x440 x439 x438 x437 x436 x435 x434 -x433 -x432 -x431 -x430 -x429 -x428 -x427 -x426 -x425 -x424 -x423 -x422 x421 -x420 -x419 x418 x417 x416 x415 x414 x413 x412 x411 x410 x409 x408 x407 -x406 -x405 -x404 -x403 -x402 -x401 -x400 -x399 -x398 -x397 x396 -x395 -x394 -x393 x392 x391 -x390 -x389 x388 x387 x386 x385 x384 x383 x382 x381 x380 x379 x378 x377 x376 x375 x374 x373 -x372 -x371 -x370 -x369 -x368 -x367 x366 -x365 -x364 -x363 x362 x361 x360 -x359 -x358 x357 x356 x355 x354 -x353 x352 x351 x350 x349 x348 x347 x346 x345 x344 x343 x342 x341 x340 x339 x338 x337 x336 x335 x334 x333 -x332 -x331 -x330 x329 -x328 -x327 x326 x325 -x324 -x323 x322 x321 x320 x319 -x318 x317 x316 x315 x314 x313 -x312 x311 x310 x309 x308 x307 x306 x305 x304 x303 x302 x301 x300 x299 x298 x297 x296 x295 x294 x293 x292 x291 x290 x289 x288 x287 x286 -x285 x284 x283 x282 x281 x280 x279 x278 x277 x276 x275 x274 x273 x272 x271 x270 x269 x268 x267 x266 x265 x264 x263 x262 x261 x260 x259 x258 x257 x256 x255 x254 x253 x252 x251 x250 x249 x248 x247 x246 x245 x244 x243 x242 x241 x240 x239 x238 x237 x236 x235 x234 x233 x232 x231 x230 x229 x228 x227 x226 x225 x224 x223 x222 x221 x220 x219 x218 x217 x216 x215 x214 x213 x212 x211 x210 x209 x208 x207 x206 x205 x204 x203 x202 x201 x200 x199 x198 x197 x196 x195 x194 x193 x192 x191 x190 x189 x188 x187 x186 x185 x184 x183 x182 x181 x180 x179 x178 x177 x176 x175 x174 x173 x172 x171 x170 x169 x168 x167 x166 x165 x164 x163 x162 x161 x160 x159 x158 x157 x156 x155 x154 x153 x152 x151 x150 x149 x148 x147 x146 x145 x144 x143 -x142 x141 x140 x139 x138 x137 x136 x135 x134 x133 x132 x131 x130 x129 x128 x127 x126 x125 x124 x123 x122 x121 x120 x119 x118 x117 x116 x115 x114 x113 x112 x111 x110 x109 x108 x107 x106 x105 x104 x103 x102 x101 x100 x99 x98 x97 x96 x95 x94 x93 x92 x91 x90 x89 x88 x87 x86 x85 x84 x83 x82 x81 x80 x79 x78

Verifier Data 

OK	76

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3703663-1338095060/watcher-3703663-1338095060 -o /tmp/evaluation-result-3703663-1338095060/solver-3703663-1338095060 -C 1800 -W 1900 -M 15500 pwbo2.0 -file-format=opb -verbosity=0 HOME/instance-3703663-1338095060.opb 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.81 3.97 3.95 4/178 21750
/proc/meminfo: memFree=31473156/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=4924 CPUtime=0 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) R 21748 21750 19605 0 -1 4202496 438 0 0 0 0 0 0 0 20 0 1 0 275631711 5042176 390 33554432000 4194304 5965102 140736607990176 140736607989592 5137146 0 0 4096 16387 0 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 1231 390 169 433 0 677 0

[startup+0.083537 s]
/proc/loadavg: 3.81 3.97 3.95 4/178 21750
/proc/meminfo: memFree=31473156/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=169348 CPUtime=0.1 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 3673 0 0 0 10 0 0 0 20 0 3 0 275631711 173412352 3505 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 42337 3505 195 433 0 41901 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 169348

[startup+0.10026 s]
/proc/loadavg: 3.81 3.97 3.95 4/178 21750
/proc/meminfo: memFree=31473156/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=169348 CPUtime=0.13 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 3678 0 0 0 13 0 0 0 20 0 3 0 275631711 173412352 3510 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 42337 3510 195 433 0 41901 0
Current children cumulated CPU time (s) 0.13
Current children cumulated vsize (KiB) 169348

[startup+0.300155 s]
/proc/loadavg: 3.81 3.97 3.95 4/178 21750
/proc/meminfo: memFree=31473156/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=170336 CPUtime=0.54 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 4102 0 0 0 53 1 0 0 20 0 3 0 275631711 174424064 3934 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 42584 3934 208 433 0 42148 0
Current children cumulated CPU time (s) 0.54
Current children cumulated vsize (KiB) 170336

[startup+0.700164 s]
/proc/loadavg: 3.81 3.97 3.95 4/178 21750
/proc/meminfo: memFree=31473156/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=170336 CPUtime=1.34 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 4268 0 0 0 133 1 0 0 20 0 3 0 275631711 174424064 4100 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 42584 4100 208 433 0 42148 0
Current children cumulated CPU time (s) 1.34
Current children cumulated vsize (KiB) 170336

[startup+1.50034 s]
/proc/loadavg: 3.81 3.97 3.95 5/181 21753
/proc/meminfo: memFree=31457756/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=170336 CPUtime=2.93 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 4599 0 0 0 292 1 0 0 20 0 3 0 275631711 174424064 4431 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 42584 4431 208 433 0 42148 0
[pid=21750/tid=21752] ppid=21748 vsize=170336 CPUtime=1.43 cores=1,3,5,7
/proc/21750/task/21752/stat : 21752 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 1313 0 0 0 143 0 0 0 20 0 3 0 275631716 174424064 4431 33554432000 4194304 5965102 140736607990176 139723862083624 4404410 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=21750/tid=21753] ppid=21748 vsize=170336 CPUtime=1.43 cores=1,3,5,7
/proc/21750/task/21753/stat : 21753 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 662 0 0 0 143 0 0 0 20 0 3 0 275631716 174424064 4431 33554432000 4194304 5965102 140736607990176 139723851594088 4222464 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 2.93
Current children cumulated vsize (KiB) 170336

[startup+3.10033 s]
/proc/loadavg: 3.81 3.97 3.95 5/181 21753
/proc/meminfo: memFree=31454640/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=170336 CPUtime=6.14 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 5165 0 0 0 612 2 0 0 20 0 3 0 275631711 174424064 4997 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 42584 4997 208 433 0 42148 0
[pid=21750/tid=21752] ppid=21748 vsize=170336 CPUtime=3.02 cores=1,3,5,7
/proc/21750/task/21752/stat : 21752 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 1681 0 0 0 301 1 0 0 20 0 3 0 275631716 174424064 4997 33554432000 4194304 5965102 140736607990176 139723862083624 4404761 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=21750/tid=21753] ppid=21748 vsize=170336 CPUtime=3.03 cores=1,3,5,7
/proc/21750/task/21753/stat : 21753 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 860 0 0 0 303 0 0 0 20 0 3 0 275631716 174424064 4997 33554432000 4194304 5965102 140736607990176 139723851594088 4404726 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 6.14
Current children cumulated vsize (KiB) 170336

[startup+6.30034 s]
/proc/loadavg: 3.83 3.97 3.95 5/181 21753
/proc/meminfo: memFree=31450920/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=170480 CPUtime=12.53 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 5667 0 0 0 1251 2 0 0 20 0 3 0 275631711 174571520 5499 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 42620 5499 208 433 0 42184 0
[pid=21750/tid=21752] ppid=21748 vsize=170480 CPUtime=6.21 cores=1,3,5,7
/proc/21750/task/21752/stat : 21752 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 2071 0 0 0 620 1 0 0 20 0 3 0 275631716 174571520 5499 33554432000 4194304 5965102 140736607990176 139723862083384 5137076 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=21750/tid=21753] ppid=21748 vsize=170480 CPUtime=6.22 cores=1,3,5,7
/proc/21750/task/21753/stat : 21753 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 972 0 0 0 622 0 0 0 20 0 3 0 275631716 174571520 5499 33554432000 4194304 5965102 140736607990176 139723851593848 4404416 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 12.53
Current children cumulated vsize (KiB) 170480

[startup+12.7004 s]
/proc/loadavg: 3.84 3.97 3.95 5/181 21753
/proc/meminfo: memFree=31444968/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=170696 CPUtime=25.33 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 6611 0 0 0 2529 4 0 0 20 0 3 0 275631711 174792704 6443 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 42674 6443 208 433 0 42238 0
[pid=21750/tid=21752] ppid=21748 vsize=170696 CPUtime=12.6 cores=1,3,5,7
/proc/21750/task/21752/stat : 21752 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 2745 0 0 0 1258 2 0 0 20 0 3 0 275631716 174792704 6443 33554432000 4194304 5965102 140736607990176 139723862083384 4404416 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=21750/tid=21753] ppid=21748 vsize=170696 CPUtime=12.6 cores=1,3,5,7
/proc/21750/task/21753/stat : 21753 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 1242 0 0 0 1259 1 0 0 20 0 3 0 275631716 174792704 6443 33554432000 4194304 5965102 140736607990176 139723851594040 4222304 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 25.33
Current children cumulated vsize (KiB) 170696

[startup+25.5004 s]
/proc/loadavg: 3.88 3.97 3.95 5/181 21753
/proc/meminfo: memFree=31435776/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=170876 CPUtime=50.92 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 8290 0 0 0 5087 5 0 0 20 0 3 0 275631711 174977024 8122 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 42719 8122 208 433 0 42283 0
[pid=21750/tid=21752] ppid=21748 vsize=170876 CPUtime=25.37 cores=1,3,5,7
/proc/21750/task/21752/stat : 21752 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 4069 0 0 0 2535 2 0 0 20 0 3 0 275631716 174977024 8122 33554432000 4194304 5965102 140736607990176 139723862083624 4220672 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=21750/tid=21753] ppid=21748 vsize=170876 CPUtime=25.37 cores=1,3,5,7
/proc/21750/task/21753/stat : 21753 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 1597 0 0 0 2535 2 0 0 20 0 3 0 275631716 174977024 8122 33554432000 4194304 5965102 140736607990176 139723851594040 4404410 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 50.92
Current children cumulated vsize (KiB) 170876

[startup+51.1004 s]
/proc/loadavg: 3.92 3.97 3.95 5/181 21753
/proc/meminfo: memFree=31417076/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=171036 CPUtime=102.11 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 10914 0 0 0 10203 8 0 0 20 0 3 0 275631711 175140864 10746 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 42759 10746 208 433 0 42323 0
[pid=21750/tid=21752] ppid=21748 vsize=171036 CPUtime=50.9 cores=1,3,5,7
/proc/21750/task/21752/stat : 21752 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 6250 0 0 0 5085 5 0 0 20 0 3 0 275631716 175140864 10746 33554432000 4194304 5965102 140736607990176 139723862083624 4222314 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=21750/tid=21753] ppid=21748 vsize=171036 CPUtime=50.9 cores=1,3,5,7
/proc/21750/task/21753/stat : 21753 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 2040 0 0 0 5088 2 0 0 20 0 3 0 275631716 175140864 10746 33554432000 4194304 5965102 140736607990176 139723851593848 4401871 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 102.11
Current children cumulated vsize (KiB) 171036

[startup+102.307 s]
/proc/loadavg: 3.96 3.97 3.95 5/181 21753
/proc/meminfo: memFree=31383296/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=171864 CPUtime=204.49 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 15360 0 0 0 20433 16 0 0 20 0 3 0 275631711 175988736 15585 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 42966 15585 208 433 0 42530 0
[pid=21750/tid=21752] ppid=21748 vsize=171864 CPUtime=101.96 cores=1,3,5,7
/proc/21750/task/21752/stat : 21752 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 9217 0 0 0 10187 9 0 0 20 0 3 0 275631716 175988736 15585 33554432000 4194304 5965102 140736607990176 139723862083576 4394002 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=21750/tid=21753] ppid=21748 vsize=171864 CPUtime=101.96 cores=1,3,5,7
/proc/21750/task/21753/stat : 21753 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 3519 0 0 0 10188 8 0 0 20 0 3 0 275631716 175988736 15585 33554432000 4194304 5965102 140736607990176 139723851594040 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 204.49
Current children cumulated vsize (KiB) 171864

[startup+162.3 s]
/proc/loadavg: 3.98 3.97 3.95 5/181 21753
/proc/meminfo: memFree=31359600/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=171060 CPUtime=324.45 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 18278 0 0 0 32423 22 0 0 20 0 3 0 275631711 175165440 19409 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 42765 19409 209 433 0 42329 0
[pid=21750/tid=21752] ppid=21748 vsize=171060 CPUtime=161.8 cores=1,3,5,7
/proc/21750/task/21752/stat : 21752 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 10512 0 0 0 16168 12 0 0 20 0 3 0 275631716 175165440 19409 33554432000 4194304 5965102 140736607990176 139723862083680 4400960 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=21750/tid=21753] ppid=21748 vsize=171060 CPUtime=161.8 cores=1,3,5,7
/proc/21750/task/21753/stat : 21753 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 5142 0 0 0 16167 13 0 0 20 0 3 0 275631716 175165440 19409 33554432000 4194304 5965102 140736607990176 139723851594088 4404767 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 324.45
Current children cumulated vsize (KiB) 171060

[startup+222.3 s]
/proc/loadavg: 3.99 3.97 3.95 5/181 21753
/proc/meminfo: memFree=31341596/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=171060 CPUtime=444.43 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 19220 0 0 0 44419 24 0 0 20 0 3 0 275631711 175165440 20352 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 42765 20352 209 433 0 42329 0
[pid=21750/tid=21752] ppid=21748 vsize=171060 CPUtime=221.64 cores=1,3,5,7
/proc/21750/task/21752/stat : 21752 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 10562 0 0 0 22150 14 0 0 20 0 3 0 275631716 175165440 20352 33554432000 4194304 5965102 140736607990176 139723862083384 4404767 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=21750/tid=21753] ppid=21748 vsize=171060 CPUtime=221.63 cores=1,3,5,7
/proc/21750/task/21753/stat : 21753 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 6034 0 0 0 22148 15 0 0 20 0 3 0 275631716 175165440 20352 33554432000 4194304 5965102 140736607990176 139723851594040 4222304 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 444.43
Current children cumulated vsize (KiB) 171060

[startup+282.3 s]
/proc/loadavg: 3.99 3.97 3.95 5/181 21754
/proc/meminfo: memFree=31331536/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=171060 CPUtime=564.39 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 20346 0 0 0 56414 25 0 0 20 0 3 0 275631711 175165440 21538 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 42765 21538 209 433 0 42329 0
[pid=21750/tid=21752] ppid=21748 vsize=171060 CPUtime=281.49 cores=1,3,5,7
/proc/21750/task/21752/stat : 21752 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 10858 0 0 0 28134 15 0 0 20 0 3 0 275631716 175165440 21538 33554432000 4194304 5965102 140736607990176 139723862083384 4400960 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=21750/tid=21753] ppid=21748 vsize=171060 CPUtime=281.47 cores=1,3,5,7
/proc/21750/task/21753/stat : 21753 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 6864 0 0 0 28130 17 0 0 20 0 3 0 275631716 175165440 21538 33554432000 4194304 5965102 140736607990176 139723851593952 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 564.39
Current children cumulated vsize (KiB) 171060

[startup+342.3 s]
/proc/loadavg: 3.99 3.97 3.95 5/182 21757
/proc/meminfo: memFree=31315712/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=171060 CPUtime=684.37 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 23449 0 0 0 68405 32 0 0 20 0 3 0 275631711 175165440 24641 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 42765 24641 209 433 0 42329 0

################
# More data... #
################

[pid=21750/tid=21753] ppid=21748 vsize=171060 CPUtime=341.31 cores=1,3,5,7
/proc/21750/task/21753/stat : 21753 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 6864 0 0 0 34113 18 0 0 20 0 3 0 275631716 175165440 24641 33554432000 4194304 5965102 140736607990176 139723851593952 4404410 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 684.37
Current children cumulated vsize (KiB) 171060

[startup+402.3 s]
/proc/loadavg: 3.99 3.97 3.95 5/181 21757
/proc/meminfo: memFree=31300820/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=236596 CPUtime=804.34 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 26175 0 0 0 80397 37 0 0 20 0 3 0 275631711 242274304 27374 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 59149 27374 209 433 0 58713 0
[pid=21750/tid=21752] ppid=21748 vsize=236596 CPUtime=401.18 cores=1,3,5,7
/proc/21750/task/21752/stat : 21752 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 16685 0 0 0 40090 28 0 0 20 0 3 0 275631716 242274304 27374 33554432000 4194304 5965102 140736607990176 139723862083624 4401143 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=21750/tid=21753] ppid=21748 vsize=236596 CPUtime=401.15 cores=1,3,5,7
/proc/21750/task/21753/stat : 21753 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 6866 0 0 0 40096 19 0 0 20 0 3 0 275631716 242274304 27374 33554432000 4194304 5965102 140736607990176 139723851593952 4222314 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 804.34
Current children cumulated vsize (KiB) 236596

[startup+462.3 s]
/proc/loadavg: 3.99 3.97 3.95 5/181 21757
/proc/meminfo: memFree=31279848/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=236596 CPUtime=924.31 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 28526 0 0 0 92389 42 0 0 20 0 3 0 275631711 242274304 29781 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 59149 29781 209 433 0 58713 0
[pid=21750/tid=21752] ppid=21748 vsize=236596 CPUtime=461.01 cores=1,3,5,7
/proc/21750/task/21752/stat : 21752 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 19034 0 0 0 46067 34 0 0 20 0 3 0 275631716 242274304 29781 33554432000 4194304 5965102 140736607990176 139723862083384 4404738 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=21750/tid=21753] ppid=21748 vsize=236596 CPUtime=460.99 cores=1,3,5,7
/proc/21750/task/21753/stat : 21753 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 6868 0 0 0 46079 20 0 0 20 0 3 0 275631716 242274304 29781 33554432000 4194304 5965102 140736607990176 139723851593952 4222416 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 924.31
Current children cumulated vsize (KiB) 236596

[startup+522.3 s]
/proc/loadavg: 3.99 3.97 3.95 5/181 21757
/proc/meminfo: memFree=31271220/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=236596 CPUtime=1044.28 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 30510 0 0 0 104384 44 0 0 20 0 3 0 275631711 242274304 31558 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 59149 31558 209 433 0 58713 0
[pid=21750/tid=21752] ppid=21748 vsize=236596 CPUtime=520.86 cores=1,3,5,7
/proc/21750/task/21752/stat : 21752 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 20967 0 0 0 52049 37 0 0 20 0 3 0 275631716 242274304 31558 33554432000 4194304 5965102 140736607990176 139723862083680 4404748 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=21750/tid=21753] ppid=21748 vsize=236596 CPUtime=520.83 cores=1,3,5,7
/proc/21750/task/21753/stat : 21753 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 6919 0 0 0 52063 20 0 0 20 0 3 0 275631716 242274304 31558 33554432000 4194304 5965102 140736607990176 139723851593848 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1044.28
Current children cumulated vsize (KiB) 236596

[startup+582.3 s]
/proc/loadavg: 3.99 3.97 3.95 5/181 21758
/proc/meminfo: memFree=31264568/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=236596 CPUtime=1164.25 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 31118 0 0 0 116379 46 0 0 20 0 3 0 275631711 242274304 32175 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 59149 32175 209 433 0 58713 0
[pid=21750/tid=21752] ppid=21748 vsize=236596 CPUtime=580.7 cores=1,3,5,7
/proc/21750/task/21752/stat : 21752 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 21009 0 0 0 58032 38 0 0 20 0 3 0 275631716 242274304 32175 33554432000 4194304 5965102 140736607990176 139723862083680 4404748 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=21750/tid=21753] ppid=21748 vsize=236596 CPUtime=580.67 cores=1,3,5,7
/proc/21750/task/21753/stat : 21753 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 7485 0 0 0 58044 23 0 0 20 0 3 0 275631716 242274304 32175 33554432000 4194304 5965102 140736607990176 139723851593848 4404410 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1164.25
Current children cumulated vsize (KiB) 236596

[startup+642.3 s]
/proc/loadavg: 3.99 3.97 3.95 5/181 21758
/proc/meminfo: memFree=31254200/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=236956 CPUtime=1284.23 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 32942 0 0 0 128374 49 0 0 20 0 3 0 275631711 242642944 34128 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 59239 34128 209 433 0 58803 0
[pid=21750/tid=21752] ppid=21748 vsize=236956 CPUtime=640.55 cores=1,3,5,7
/proc/21750/task/21752/stat : 21752 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 21013 0 0 0 64016 39 0 0 20 0 3 0 275631716 242642944 34128 33554432000 4194304 5965102 140736607990176 139723862083680 4222320 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=21750/tid=21753] ppid=21748 vsize=236956 CPUtime=640.51 cores=1,3,5,7
/proc/21750/task/21753/stat : 21753 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 9305 0 0 0 64024 27 0 0 20 0 3 0 275631716 242642944 34128 33554432000 4194304 5965102 140736607990176 139723851594040 4222320 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1284.23
Current children cumulated vsize (KiB) 236956

[startup+702.3 s]
/proc/loadavg: 3.99 3.97 3.95 5/181 21758
/proc/meminfo: memFree=31246264/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=236956 CPUtime=1404.2 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 32943 0 0 0 140371 49 0 0 20 0 3 0 275631711 242642944 34129 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 59239 34129 209 433 0 58803 0
[pid=21750/tid=21752] ppid=21748 vsize=236956 CPUtime=700.39 cores=1,3,5,7
/proc/21750/task/21752/stat : 21752 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 21014 0 0 0 69999 40 0 0 20 0 3 0 275631716 242642944 34129 33554432000 4194304 5965102 140736607990176 139723862083680 4404416 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=21750/tid=21753] ppid=21748 vsize=236956 CPUtime=700.35 cores=1,3,5,7
/proc/21750/task/21753/stat : 21753 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 9305 0 0 0 70007 28 0 0 20 0 3 0 275631716 242642944 34129 33554432000 4194304 5965102 140736607990176 139723851594040 5144430 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1404.2
Current children cumulated vsize (KiB) 236956

[startup+762.307 s]
/proc/loadavg: 3.99 3.97 3.95 5/181 21758
/proc/meminfo: memFree=31243832/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=236956 CPUtime=1524.19 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 33289 0 0 0 152369 50 0 0 20 0 3 0 275631711 242642944 34474 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 59239 34474 209 433 0 58803 0
[pid=21750/tid=21752] ppid=21748 vsize=236956 CPUtime=760.24 cores=1,3,5,7
/proc/21750/task/21752/stat : 21752 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 21014 0 0 0 75983 41 0 0 20 0 3 0 275631716 242642944 34474 33554432000 4194304 5965102 140736607990176 139723862083680 4221120 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=21750/tid=21753] ppid=21748 vsize=236956 CPUtime=760.19 cores=1,3,5,7
/proc/21750/task/21753/stat : 21753 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 9651 0 0 0 75990 29 0 0 20 0 3 0 275631716 242642944 34474 33554432000 4194304 5965102 140736607990176 139723851593912 4222314 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1524.19
Current children cumulated vsize (KiB) 236956

[startup+822.3 s]
/proc/loadavg: 3.99 3.97 3.95 5/181 21758
/proc/meminfo: memFree=31232668/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=236956 CPUtime=1644.15 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 33548 0 0 0 164365 50 0 0 20 0 3 0 275631711 242642944 34737 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 59239 34737 209 433 0 58803 0
[pid=21750/tid=21752] ppid=21748 vsize=236956 CPUtime=820.08 cores=1,3,5,7
/proc/21750/task/21752/stat : 21752 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 21023 0 0 0 81966 42 0 0 20 0 3 0 275631716 242642944 34737 33554432000 4194304 5965102 140736607990176 139723862083680 4400960 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=21750/tid=21753] ppid=21748 vsize=236956 CPUtime=820.03 cores=1,3,5,7
/proc/21750/task/21753/stat : 21753 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 9901 0 0 0 81972 31 0 0 20 0 3 0 275631716 242642944 34737 33554432000 4194304 5965102 140736607990176 139723851593848 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1644.15
Current children cumulated vsize (KiB) 236956

[startup+882.3 s]
/proc/loadavg: 3.94 3.96 3.94 5/181 21800
/proc/meminfo: memFree=31132124/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=236956 CPUtime=1764.12 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 35160 0 0 0 176360 52 0 0 20 0 3 0 275631711 242642944 36373 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 59239 36373 209 433 0 58803 0
[pid=21750/tid=21752] ppid=21748 vsize=236956 CPUtime=879.92 cores=1,3,5,7
/proc/21750/task/21752/stat : 21752 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 22634 0 0 0 87947 45 0 0 20 0 3 0 275631716 242642944 36373 33554432000 4194304 5965102 140736607990176 139723862083576 4404410 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=21750/tid=21753] ppid=21748 vsize=236956 CPUtime=879.87 cores=1,3,5,7
/proc/21750/task/21753/stat : 21753 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 9902 0 0 0 87955 32 0 0 20 0 3 0 275631716 242642944 36373 33554432000 4194304 5965102 140736607990176 139723851593848 4404416 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1764.12
Current children cumulated vsize (KiB) 236956



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+900.3 s]
/proc/loadavg: 4.03 3.97 3.95 5/181 21800
/proc/meminfo: memFree=31023540/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=236956 CPUtime=1800.12 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 35177 0 0 0 179960 52 0 0 20 0 3 0 275631711 242642944 36390 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 59239 36390 209 433 0 58803 0
[pid=21750/tid=21752] ppid=21748 vsize=236956 CPUtime=897.88 cores=1,3,5,7
/proc/21750/task/21752/stat : 21752 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 22651 0 0 0 89743 45 0 0 20 0 3 0 275631716 242642944 36390 33554432000 4194304 5965102 140736607990176 139723862083576 4221120 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=21750/tid=21753] ppid=21748 vsize=236956 CPUtime=897.82 cores=1,3,5,7
/proc/21750/task/21753/stat : 21753 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 9902 0 0 0 89750 32 0 0 20 0 3 0 275631716 242642944 36390 33554432000 4194304 5965102 140736607990176 139723851593848 4404410 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1800.12
Current children cumulated vsize (KiB) 236956

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+900.3 s]
/proc/loadavg: 4.03 3.97 3.95 5/181 21800
/proc/meminfo: memFree=31023540/32873844 swapFree=7112/7112
[pid=21750] ppid=21748 vsize=236956 CPUtime=1800.12 cores=1,3,5,7
/proc/21750/stat : 21750 (pwbo2.0) S 21748 21750 19605 0 -1 4202496 35177 0 0 0 179960 52 0 0 20 0 3 0 275631711 242642944 36390 33554432000 4194304 5965102 140736607990176 140736607989696 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/21750/statm: 59239 36390 209 433 0 58803 0
[pid=21750/tid=21752] ppid=21748 vsize=236956 CPUtime=897.88 cores=1,3,5,7
/proc/21750/task/21752/stat : 21752 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 22651 0 0 0 89743 45 0 0 20 0 3 0 275631716 242642944 36390 33554432000 4194304 5965102 140736607990176 139723862083576 4221120 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=21750/tid=21753] ppid=21748 vsize=236956 CPUtime=897.82 cores=1,3,5,7
/proc/21750/task/21753/stat : 21753 (pwbo2.0) R 21748 21750 19605 0 -1 4202560 9902 0 0 0 89750 32 0 0 20 0 3 0 275631716 242642944 36390 33554432000 4194304 5965102 140736607990176 139723851593848 4404410 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1800.12
Current children cumulated vsize (KiB) 236956

Child status: 0

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 21750 and gives
#  childrusage.ru_utime.tv_sec=1795
#  childrusage.ru_utime.tv_usec=118
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=795879
# CPU time returned by wait4() is 1795.8
# while last known CPU time is 1800.12
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 900.314
CPU time (s): 1800.12
CPU user time (s): 1799.6
CPU system time (s): 0.52
CPU usage (%): 199.944
Max. virtual memory (cumulated for all children) (KiB): 237680

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1795
system time used= 0.795879
maximum resident set size= 145568
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 35179
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 128
involuntary context switches= 2132

runsolver used 2.18967 second user time and 5.33519 second system time

The end

Launcher Data 

Begin job on node119 at 2012-05-27 07:04:20
IDJOB=3703663
IDBENCH=2940
IDSOLVER=2279
FILE ID=node119/3703663-1338095060
RUNJOBID= node119-1338085105-19622
PBS_JOBID= 14624426
Free space on /tmp= 71604 MiB

SOLVER NAME= pwbo 2.0
BENCH NAME= PB06/final/normalized-PB06/OPT-SMALLINT/submitted-PB06/manquiho/bounded_golomb_rulers/normalized-bogr_11.opb
COMMAND LINE= pwbo2.0 -file-format=opb -verbosity=0 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3703663-1338095060/watcher-3703663-1338095060 -o /tmp/evaluation-result-3703663-1338095060/solver-3703663-1338095060 -C 1800 -W 1900 -M 15500  pwbo2.0 -file-format=opb -verbosity=0 HOME/instance-3703663-1338095060.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= c19ad978a977652a94fc03b8ce63204a
RANDOM SEED=760517288

node119.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.88
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        31473048 kB
Buffers:          257052 kB
Cached:           270488 kB
SwapCached:         1776 kB
Active:           545832 kB
Inactive:          58692 kB
Active(anon):      77424 kB
Inactive(anon):     1972 kB
Active(file):     468408 kB
Inactive(file):    56720 kB
Unevictable:        7112 kB
Mlocked:            7112 kB
SwapTotal:      67108856 kB
SwapFree:       67100168 kB
Dirty:              5832 kB
Writeback:            12 kB
AnonPages:         82904 kB
Mapped:            12552 kB
Shmem:                 8 kB
Slab:             655672 kB
SReclaimable:      97928 kB
SUnreclaim:       557744 kB
KernelStack:        1472 kB
PageTables:         3772 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     193280 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:      2048 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 71596 MiB
End job on node119 at 2012-05-27 07:19:23