Trace number 34415

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds, and are wall clock time (not CPU time).

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerObjective functionCPU timeWall clock time
PB-smodels 1.28SAT (TO)307 1800.72 1801.26

General information on the benchmark

Namenormalized-PB06/OPT-SMALLINT/submitted-PB05/
manquinho/primes-dimacs-cnf/normalized-ii8a4.opb
MD5SUM39250f205953aedd83d7a2bcba6bda16
Bench CategoryOPT-SMALLINT (optimisation, small integers)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark283
Best CPU time to get the best result obtained on this benchmark1800.03
Has Objective FunctionYES
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function 283
Optimality of the best value was proved NO
Number of variables792
Total number of constraints3194
Number of constraints which are clauses3194
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint2
Maximum length of a constraint8
Number of terms in the objective function 792
Biggest coefficient in the objective function 1
Number of bits for the biggest coefficient in the objective function 1
Sum of the numbers in the objective function 792
Number of bits of the sum of numbers in the objective function 10
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 792
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data (download as text)

0.00	c Running pbsmodels-v1.28
1.17	o 307
1801.24	sh: line 1: 29159 Done                    pbsmodels.trans ROOT/tmp/node56/34415-1154047907/instance-34415-1154047907.opb ROOT/tmp/node56/34415-1154047907
1801.24	     29160 Terminated              | smodels -seed 511467414
1801.24	     29161 Terminated              | pbout.pl ROOT/tmp/node56/34415-1154047907/instance-34415-1154047907.opb ROOT/tmp/node56/34415-1154047907
1801.25	s SATISFIABLE
1801.25	v x1 x3 x5 x7 x9 x11 x14 x15 x17 x19 x21 x23 x25 x28 x29 x31 x33 x35 x37 x39 x41 x43 x46 x47 x49 x51 x53 x55 x57 x60 x61 x63 x65 x67 x69 x71
1801.25	v x73 x75 x78 x79 x81 x83 x85 x87 x89 x92 x93 x95 x98 x99 x102 x103 x105 x107 x109 x111 x113 x115 x117 x120 x121 x127 x130 x131 x134
1801.25	v x135 x137 x140 x141 x143 x145 x147 x149 x151 x153 x155 x157 x159 x161 x163 x165 x167 x169 x172 x174 x175 x177 x179 x181 x183 x185
1801.25	v x187 x189 x191 x195 x200 x202 x207 x212 x214 x216 x219 x230 x232 x234 x236 x237 x240 x248 x250 x251 x254 x256 x258 x260 x262 x263
1801.25	v x267 x272 x274 x277 x284 x286 x293 x296 x298 x300 x305 x308 x310 x312 x314 x316 x318 x320 x322 x323 x329 x332 x334 x338 x340 x342
1801.25	v x344 x345 x348 x351 x356 x358 x360 x365 x368 x370 x377 x380 x382 x386 x388 x390 x392 x394 x395 x397 x404 x406 x416 x419 x421 x428
1801.25	v x430 x434 x436 x438 x440 x442 x443 x449 x452 x454 x456 x459 x464 x466 x468 x476 x477 x485 x488 x490 x492 x494 x496 x498 x499 x504
1801.25	v x506 x508 x510 x512 x514 x515 x518 x520 x522 x524 x526 x527 x530 x532 x534 x536 x538 x539 x543 x548 x550 x552 x557 x560 x562 x564
1801.25	v x566 x568 x570 x572 x575 x578 x580 x582 x584 x586 x587 x591 x596 x598 x600 x602 x604 x606 x608 x611 x613 x620 x622 x624 x626 x628
1801.25	v x630 x632 x633 x636 x639 x644 x646 x651 x656 x658 x665 x668 x670 x672 x675 x680 x682 x684 x686 x688 x690 x692 x695 x697 x704 x706
1801.25	v x708 x710 x712 x714 x716 x718 x719 x721 x728 x730 x732 x733 x740 x742 x746 x748 x750 x751 x754 x756 x761 x764 x766 x768 x769 x776
1801.25	v x778 x780 x782 x784 x786 x788 x789 x792 -x2 -x4 -x6 -x8 -x10 -x12 -x13 -x16 -x18 -x20 -x22 -x24 -x26 -x27 -x30 -x32 -x34 -x36 -x38
1801.25	v -x40 -x42 -x44 -x45 -x48 -x50 -x52 -x54 -x56 -x58 -x59 -x62 -x64 -x66 -x68 -x70 -x72 -x74 -x76 -x77 -x80 -x82 -x84 -x86 -x88 -x90
1801.25	v -x91 -x94 -x96 -x97 -x100 -x101 -x104 -x106 -x108 -x110 -x112 -x114 -x116 -x118 -x119 -x122 -x123 -x124 -x125 -x126 -x128
1801.25	v -x129 -x132 -x133 -x136 -x138 -x139 -x142 -x144 -x146 -x148 -x150 -x152 -x154 -x156 -x158 -x160 -x162 -x164 -x166 -x168 -x170
1801.25	v -x171 -x173 -x176 -x178 -x180 -x182 -x184 -x186 -x188 -x190 -x192 -x193 -x194 -x196 -x197 -x198 -x199 -x201 -x203 -x204 -x205
1801.25	v -x206 -x208 -x209 -x210 -x211 -x213 -x215 -x217 -x218 -x220 -x221 -x222 -x223 -x224 -x225 -x226 -x227 -x228 -x229 -x231 -x233
1801.25	v -x235 -x238 -x239 -x241 -x242 -x243 -x244 -x245 -x246 -x247 -x249 -x252 -x253 -x255 -x257 -x259 -x261 -x264 -x265 -x266 -x268
1801.25	v -x269 -x270 -x271 -x273 -x275 -x276 -x278 -x279 -x280 -x281 -x282 -x283 -x285 -x287 -x288 -x289 -x290 -x291 -x292 -x294 -x295
1801.25	v -x297 -x299 -x301 -x302 -x303 -x304 -x306 -x307 -x309 -x311 -x313 -x315 -x317 -x319 -x321 -x324 -x325 -x326 -x327 -x328 -x330
1801.25	v -x331 -x333 -x335 -x336 -x337 -x339 -x341 -x343 -x346 -x347 -x349 -x350 -x352 -x353 -x354 -x355 -x357 -x359 -x361 -x362 -x363
1801.25	v -x364 -x366 -x367 -x369 -x371 -x372 -x373 -x374 -x375 -x376 -x378 -x379 -x381 -x383 -x384 -x385 -x387 -x389 -x391 -x393 -x396
1801.25	v -x398 -x399 -x400 -x401 -x402 -x403 -x405 -x407 -x408 -x409 -x410 -x411 -x412 -x413 -x414 -x415 -x417 -x418 -x420 -x422 -x423
1801.25	v -x424 -x425 -x426 -x427 -x429 -x431 -x432 -x433 -x435 -x437 -x439 -x441 -x444 -x445 -x446 -x447 -x448 -x450 -x451 -x453 -x455
1801.25	v -x457 -x458 -x460 -x461 -x462 -x463 -x465 -x467 -x469 -x470 -x471 -x472 -x473 -x474 -x475 -x478 -x479 -x480 -x481 -x482 -x483
1801.25	v -x484 -x486 -x487 -x489 -x491 -x493 -x495 -x497 -x500 -x501 -x502 -x503 -x505 -x507 -x509 -x511 -x513 -x516 -x517 -x519 -x521
1801.25	v -x523 -x525 -x528 -x529 -x531 -x533 -x535 -x537 -x540 -x541 -x542 -x544 -x545 -x546 -x547 -x549 -x551 -x553 -x554 -x555 -x556
1801.25	v -x558 -x559 -x561 -x563 -x565 -x567 -x569 -x571 -x573 -x574 -x576 -x577 -x579 -x581 -x583 -x585 -x588 -x589 -x590 -x592 -x593
1801.25	v -x594 -x595 -x597 -x599 -x601 -x603 -x605 -x607 -x609 -x610 -x612 -x614 -x615 -x616 -x617 -x618 -x619 -x621 -x623 -x625 -x627
1801.25	v -x629 -x631 -x634 -x635 -x637 -x638 -x640 -x641 -x642 -x643 -x645 -x647 -x648 -x649 -x650 -x652 -x653 -x654 -x655 -x657 -x659
1801.25	v -x660 -x661 -x662 -x663 -x664 -x666 -x667 -x669 -x671 -x673 -x674 -x676 -x677 -x678 -x679 -x681 -x683 -x685 -x687 -x689 -x691
1801.25	v -x693 -x694 -x696 -x698 -x699 -x700 -x701 -x702 -x703 -x705 -x707 -x709 -x711 -x713 -x715 -x717 -x720 -x722 -x723 -x724 -x725
1801.25	v -x726 -x727 -x729 -x731 -x734 -x735 -x736 -x737 -x738 -x739 -x741 -x743 -x744 -x745 -x747 -x749 -x752 -x753 -x755 -x757 -x758
1801.25	v -x759 -x760 -x762 -x763 -x765 -x767 -x770 -x771 -x772 -x773 -x774 -x775 -x777 -x779 -x781 -x783 -x785 -x787 -x790 -x791
1801.25	c Somebody sent me a SIGTERM
1801.26	smodels: no process killed
1801.26	pbsmodels.trans: no process killed

Verifier Data (download as text)

OK	307

Watcher Data (download as text)

Enforcing CPU limit (will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime (will send SIGXCPU) limit: 1830 seconds
Enforcing Stack size limit: 67108864 bytes

runsolver version 3.0.3 (c) roussel@cril.univ-artois.fr

Enforcing memory limit (will send SIGKILL): 1843200 Kb
Enforcing VSIZE limit: 1887436800 bytes
command line: runsolver --timestamp -w ROOT/results/node56/watcher-34415-1154047907 -o ROOT/results/node56/solver-34415-1154047907 -C 1800 -M 1800 -S 64 ROOT/solvers/PB/PB06final/user4/pbsmodels_solver-v1.28/runpbsmodels ROOT/tmp/node56/34415-1154047907/instance-34415-1154047907.opb 511467414 ROOT/tmp/node56/34415-1154047907 

Current StackSize limit: 67108864 bytes

/proc/loadavg: 1.04 0.99 0.99 2/64 29156
/proc/meminfo: memFree=1617592/2055920 swapFree=4182032/4192956
[pid=29156] ppid=29154 vsize=5356 CPUtime=0
/proc/29156/stat : 29156 (runpbsmodels) R 29154 29156 29107 0 -1 4194304 206 0 0 0 0 0 0 0 19 0 1 0 151660687 5484544 172 18446744073709551615 4194304 4889804 548682069168 18446744073709551615 4372176 0 0 4100 0 0 0 0 17 1 0 0
/proc/29156/statm: 1339 174 142 169 0 50 0

[startup+10.0021 s]
/proc/loadavg: 1.04 0.99 0.99 2/68 29161
/proc/meminfo: memFree=1615464/2055920 swapFree=4182032/4192956
[pid=29156] ppid=29154 vsize=5360 CPUtime=0
/proc/29156/stat : 29156 (runpbsmodels) S 29154 29156 29107 0 -1 4194304 287 0 0 0 0 0 0 0 19 0 1 0 151660687 5488640 234 18446744073709551615 4194304 4889804 548682069168 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/29156/statm: 1340 234 195 169 0 51 0
[pid=29157] ppid=29156 vsize=12268 CPUtime=0
/proc/29157/stat : 29157 (pbsmodels-v1.28) S 29156 29156 29107 0 -1 4194304 425 0 0 0 0 0 0 0 18 0 1 0 151660688 12562432 383 18446744073709551615 4194304 4206940 548682068992 18446744073709551615 253316087700 0 0 4230 16384 18446744071563356171 0 0 17 0 0 0
/proc/29157/statm: 3067 383 293 3 0 167 0
[pid=29158] ppid=29157 vsize=5356 CPUtime=0.18
/proc/29158/stat : 29158 (sh) S 29157 29156 29107 0 -1 4194304 311 572 0 0 0 0 18 0 16 0 1 0 151660689 5484544 227 18446744073709551615 4194304 4889804 548682069024 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/29158/statm: 1339 227 188 169 0 50 0
[pid=29160] ppid=29158 vsize=2964 CPUtime=9.87
/proc/29160/stat : 29160 (smodels) R 29158 29156 29107 0 -1 4194304 425 0 0 0 987 0 0 0 25 0 1 0 151660689 3035136 389 18446744073709551615 134512640 134595782 4294956656 18446744073709551615 134534209 0 0 4096 0 0 0 0 17 1 0 0
/proc/29160/statm: 741 389 212 20 0 178 0
[pid=29161] ppid=29158 vsize=12268 CPUtime=0
/proc/29161/stat : 29161 (pbout.pl) S 29158 29156 29107 0 -1 4194304 469 0 0 0 0 0 0 0 16 0 1 0 151660689 12562432 428 18446744073709551615 4194304 4206940 548682069040 18446744073709551615 253316083810 0 0 4224 0 18446744071563648864 0 0 17 0 0 0
/proc/29161/statm: 3067 428 333 3 0 167 0
Current children cumulated CPU time (s) 10.05
Current children cumulated vsize (Kb) 38216

[startup+20.0091 s]
/proc/loadavg: 1.03 0.99 0.99 2/68 29161
/proc/meminfo: memFree=1615600/2055920 swapFree=4182032/4192956
[pid=29156] ppid=29154 vsize=5360 CPUtime=0
/proc/29156/stat : 29156 (runpbsmodels) S 29154 29156 29107 0 -1 4194304 287 0 0 0 0 0 0 0 19 0 1 0 151660687 5488640 234 18446744073709551615 4194304 4889804 548682069168 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/29156/statm: 1340 234 195 169 0 51 0
[pid=29157] ppid=29156 vsize=12268 CPUtime=0
/proc/29157/stat : 29157 (pbsmodels-v1.28) S 29156 29156 29107 0 -1 4194304 425 0 0 0 0 0 0 0 18 0 1 0 151660688 12562432 383 18446744073709551615 4194304 4206940 548682068992 18446744073709551615 253316087700 0 0 4230 16384 18446744071563356171 0 0 17 0 0 0
/proc/29157/statm: 3067 383 293 3 0 167 0
[pid=29158] ppid=29157 vsize=5356 CPUtime=0.18
/proc/29158/stat : 29158 (sh) S 29157 29156 29107 0 -1 4194304 311 572 0 0 0 0 18 0 16 0 1 0 151660689 5484544 227 18446744073709551615 4194304 4889804 548682069024 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/29158/statm: 1339 227 188 169 0 50 0
[pid=29160] ppid=29158 vsize=2964 CPUtime=19.87
/proc/29160/stat : 29160 (smodels) R 29158 29156 29107 0 -1 4194304 425 0 0 0 1987 0 0 0 25 0 1 0 151660689 3035136 389 18446744073709551615 134512640 134595782 4294956656 18446744073709551615 134531114 0 0 4096 0 0 0 0 17 1 0 0
/proc/29160/statm: 741 389 212 20 0 178 0
[pid=29161] ppid=29158 vsize=12268 CPUtime=0
/proc/29161/stat : 29161 (pbout.pl) S 29158 29156 29107 0 -1 4194304 469 0 0 0 0 0 0 0 16 0 1 0 151660689 12562432 428 18446744073709551615 4194304 4206940 548682069040 18446744073709551615 253316083810 0 0 4224 0 18446744071563648864 0 0 17 0 0 0
/proc/29161/statm: 3067 428 333 3 0 167 0
Current children cumulated CPU time (s) 20.05
Current children cumulated vsize (Kb) 38216

[startup+30.0161 s]
/proc/loadavg: 1.03 0.99 0.99 2/68 29161
/proc/meminfo: memFree=1615664/2055920 swapFree=4182032/4192956
[pid=29156] ppid=29154 vsize=5360 CPUtime=0
/proc/29156/stat : 29156 (runpbsmodels) S 29154 29156 29107 0 -1 4194304 287 0 0 0 0 0 0 0 19 0 1 0 151660687 5488640 234 18446744073709551615 4194304 4889804 548682069168 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/29156/statm: 1340 234 195 169 0 51 0
[pid=29157] ppid=29156 vsize=12268 CPUtime=0
/proc/29157/stat : 29157 (pbsmodels-v1.28) S 29156 29156 29107 0 -1 4194304 425 0 0 0 0 0 0 0 18 0 1 0 151660688 12562432 383 18446744073709551615 4194304 4206940 548682068992 18446744073709551615 253316087700 0 0 4230 16384 18446744071563356171 0 0 17 0 0 0
/proc/29157/statm: 3067 383 293 3 0 167 0
[pid=29158] ppid=29157 vsize=5356 CPUtime=0.18
/proc/29158/stat : 29158 (sh) S 29157 29156 29107 0 -1 4194304 311 572 0 0 0 0 18 0 16 0 1 0 151660689 5484544 227 18446744073709551615 4194304 4889804 548682069024 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/29158/statm: 1339 227 188 169 0 50 0
[pid=29160] ppid=29158 vsize=2964 CPUtime=29.88
/proc/29160/stat : 29160 (smodels) R 29158 29156 29107 0 -1 4194304 425 0 0 0 2988 0 0 0 25 0 1 0 151660689 3035136 389 18446744073709551615 134512640 134595782 4294956656 18446744073709551615 134548950 0 0 4096 0 0 0 0 17 1 0 0
/proc/29160/statm: 741 389 212 20 0 178 0
[pid=29161] ppid=29158 vsize=12268 CPUtime=0
/proc/29161/stat : 29161 (pbout.pl) S 29158 29156 29107 0 -1 4194304 469 0 0 0 0 0 0 0 16 0 1 0 151660689 12562432 428 18446744073709551615 4194304 4206940 548682069040 18446744073709551615 253316083810 0 0 4224 0 18446744071563648864 0 0 17 0 0 0
/proc/29161/statm: 3067 428 333 3 0 167 0
Current children cumulated CPU time (s) 30.06
Current children cumulated vsize (Kb) 38216

[startup+40.0231 s]
/proc/loadavg: 1.02 0.99 0.99 2/68 29161
/proc/meminfo: memFree=1615600/2055920 swapFree=4182032/4192956
[pid=29156] ppid=29154 vsize=5360 CPUtime=0
/proc/29156/stat : 29156 (runpbsmodels) S 29154 29156 29107 0 -1 4194304 287 0 0 0 0 0 0 0 19 0 1 0 151660687 5488640 234 18446744073709551615 4194304 4889804 548682069168 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/29156/statm: 1340 234 195 169 0 51 0
[pid=29157] ppid=29156 vsize=12268 CPUtime=0
/proc/29157/stat : 29157 (pbsmodels-v1.28) S 29156 29156 29107 0 -1 4194304 425 0 0 0 0 0 0 0 18 0 1 0 151660688 12562432 383 18446744073709551615 4194304 4206940 548682068992 18446744073709551615 253316087700 0 0 4230 16384 18446744071563356171 0 0 17 0 0 0
/proc/29157/statm: 3067 383 293 3 0 167 0
[pid=29158] ppid=29157 vsize=5356 CPUtime=0.18
/proc/29158/stat : 29158 (sh) S 29157 29156 29107 0 -1 4194304 311 572 0 0 0 0 18 0 16 0 1 0 151660689 5484544 227 18446744073709551615 4194304 4889804 548682069024 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/29158/statm: 1339 227 188 169 0 50 0
[pid=29160] ppid=29158 vsize=2964 CPUtime=39.88
/proc/29160/stat : 29160 (smodels) R 29158 29156 29107 0 -1 4194304 425 0 0 0 3988 0 0 0 25 0 1 0 151660689 3035136 389 18446744073709551615 134512640 134595782 4294956656 18446744073709551615 134544745 0 0 4096 0 0 0 0 17 1 0 0
/proc/29160/statm: 741 389 212 20 0 178 0
[pid=29161] ppid=29158 vsize=12268 CPUtime=0
/proc/29161/stat : 29161 (pbout.pl) S 29158 29156 29107 0 -1 4194304 469 0 0 0 0 0 0 0 16 0 1 0 151660689 12562432 428 18446744073709551615 4194304 4206940 548682069040 18446744073709551615 253316083810 0 0 4224 0 18446744071563648864 0 0 17 0 0 0
/proc/29161/statm: 3067 428 333 3 0 167 0
Current children cumulated CPU time (s) 40.06
Current children cumulated vsize (Kb) 38216

[startup+50.0291 s]
/proc/loadavg: 1.02 0.99 0.99 2/68 29161
/proc/meminfo: memFree=1615664/2055920 swapFree=4182032/4192956
[pid=29156] ppid=29154 vsize=5360 CPUtime=0
/proc/29156/stat : 29156 (runpbsmodels) S 29154 29156 29107 0 -1 4194304 287 0 0 0 0 0 0 0 19 0 1 0 151660687 5488640 234 18446744073709551615 4194304 4889804 548682069168 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/29156/statm: 1340 234 195 169 0 51 0
[pid=29157] ppid=29156 vsize=12268 CPUtime=0
/proc/29157/stat : 29157 (pbsmodels-v1.28) S 29156 29156 29107 0 -1 4194304 425 0 0 0 0 0 0 0 18 0 1 0 151660688 12562432 383 18446744073709551615 4194304 4206940 548682068992 18446744073709551615 253316087700 0 0 4230 16384 18446744071563356171 0 0 17 0 0 0
/proc/29157/statm: 3067 383 293 3 0 167 0
[pid=29158] ppid=29157 vsize=5356 CPUtime=0.18
/proc/29158/stat : 29158 (sh) S 29157 29156 29107 0 -1 4194304 311 572 0 0 0 0 18 0 16 0 1 0 151660689 5484544 227 18446744073709551615 4194304 4889804 548682069024 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/29158/statm: 1339 227 188 169 0 50 0
[pid=29160] ppid=29158 vsize=2964 CPUtime=49.88
/proc/29160/stat : 29160 (smodels) R 29158 29156 29107 0 -1 4194304 425 0 0 0 4988 0 0 0 25 0 1 0 151660689 3035136 389 18446744073709551615 134512640 134595782 4294956656 18446744073709551615 134521138 0 0 4096 0 0 0 0 17 1 0 0
/proc/29160/statm: 741 389 212 20 0 178 0
[pid=29161] ppid=29158 vsize=12268 CPUtime=0
/proc/29161/stat : 29161 (pbout.pl) S 29158 29156 29107 0 -1 4194304 469 0 0 0 0 0 0 0 16 0 1 0 151660689 12562432 428 18446744073709551615 4194304 4206940 548682069040 18446744073709551615 253316083810 0 0 4224 0 18446744071563648864 0 0 17 0 0 0
/proc/29161/statm: 3067 428 333 3 0 167 0
Current children cumulated CPU time (s) 50.06
Current children cumulated vsize (Kb) 38216

[startup+60.0351 s]
/proc/loadavg: 1.01 0.99 0.99 2/68 29161
/proc/meminfo: memFree=1615664/2055920 swapFree=4182032/4192956
[pid=29156] ppid=29154 vsize=5360 CPUtime=0
/proc/29156/stat : 29156 (runpbsmodels) S 29154 29156 29107 0 -1 4194304 287 0 0 0 0 0 0 0 19 0 1 0 151660687 5488640 234 18446744073709551615 4194304 4889804 548682069168 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/29156/statm: 1340 234 195 169 0 51 0
[pid=29157] ppid=29156 vsize=12268 CPUtime=0
/proc/29157/stat : 29157 (pbsmodels-v1.28) S 29156 29156 29107 0 -1 4194304 425 0 0 0 0 0 0 0 18 0 1 0 151660688 12562432 383 18446744073709551615 4194304 4206940 548682068992 18446744073709551615 253316087700 0 0 4230 16384 18446744071563356171 0 0 17 0 0 0
/proc/29157/statm: 3067 383 293 3 0 167 0
[pid=29158] ppid=29157 vsize=5356 CPUtime=0.18
/proc/29158/stat : 29158 (sh) S 29157 29156 29107 0 -1 4194304 311 572 0 0 0 0 18 0 16 0 1 0 151660689 5484544 227 18446744073709551615 4194304 4889804 548682069024 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/29158/statm: 1339 227 188 169 0 50 0
[pid=29160] ppid=29158 vsize=2964 CPUtime=59.89
/proc/29160/stat : 29160 (smodels) R 29158 29156 29107 0 -1 4194304 425 0 0 0 5989 0 0 0 25 0 1 0 151660689 3035136 389 18446744073709551615 134512640 134595782 4294956656 18446744073709551615 134549104 0 0 4096 0 0 0 0 17 1 0 0
/proc/29160/statm: 741 389 212 20 0 178 0
[pid=29161] ppid=29158 vsize=12268 CPUtime=0
/proc/29161/stat : 29161 (pbout.pl) S 29158 29156 29107 0 -1 4194304 469 0 0 0 0 0 0 0 16 0 1 0 151660689 12562432 428 18446744073709551615 4194304 4206940 548682069040 18446744073709551615 253316083810 0 0 4224 0 18446744071563648864 0 0 17 0 0 0
/proc/29161/statm: 3067 428 333 3 0 167 0
Current children cumulated CPU time (s) 60.07
Current children cumulated vsize (Kb) 38216

[startup+70.0411 s]
/proc/loadavg: 1.01 0.99 0.99 2/68 29161
/proc/meminfo: memFree=1615664/2055920 swapFree=4182032/4192956
[pid=29156] ppid=29154 vsize=5360 CPUtime=0
/proc/29156/stat : 29156 (runpbsmodels) S 29154 29156 29107 0 -1 4194304 287 0 0 0 0 0 0 0 19 0 1 0 151660687 5488640 234 18446744073709551615 4194304 4889804 548682069168 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/29156/statm: 1340 234 195 169 0 51 0
[pid=29157] ppid=29156 vsize=12268 CPUtime=0
/proc/29157/stat : 29157 (pbsmodels-v1.28) S 29156 29156 29107 0 -1 4194304 425 0 0 0 0 0 0 0 18 0 1 0 151660688 12562432 383 18446744073709551615 4194304 4206940 548682068992 18446744073709551615 253316087700 0 0 4230 16384 18446744071563356171 0 0 17 0 0 0
/proc/29157/statm: 3067 383 293 3 0 167 0
[pid=29158] ppid=29157 vsize=5356 CPUtime=0.18
/proc/29158/stat : 29158 (sh) S 29157 29156 29107 0 -1 4194304 311 572 0 0 0 0 18 0 16 0 1 0 151660689 5484544 227 18446744073709551615 4194304 4889804 548682069024 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/29158/statm: 1339 227 188 169 0 50 0
[pid=29160] ppid=29158 vsize=2964 CPUtime=69.89
/proc/29160/stat : 29160 (smodels) R 29158 29156 29107 0 -1 4194304 425 0 0 0 6989 0 0 0 25 0 1 0 151660689 3035136 389 18446744073709551615 134512640 134595782 4294956656 18446744073709551615 134534245 0 0 4096 0 0 0 0 17 1 0 0
/proc/29160/statm: 741 389 212 20 0 178 0
[pid=29161] ppid=29158 vsize=12268 CPUtime=0
/proc/29161/stat : 29161 (pbout.pl) S 29158 29156 29107 0 -1 4194304 469 0 0 0 0 0 0 0 16 0 1 0 151660689 12562432 428 18446744073709551615 4194304 4206940 548682069040 18446744073709551615 253316083810 0 0 4224 0 18446744071563648864 0 0 17 0 0 0
/proc/29161/statm: 3067 428 333 3 0 167 0
Current children cumulated CPU time (s) 70.07
Current children cumulated vsize (Kb) 38216

[startup+80.048 s]
/proc/loadavg: 1.01 0.99 0.99 2/68 29161
/proc/meminfo: memFree=1615664/2055920 swapFree=4182032/4192956
[pid=29156] ppid=29154 vsize=5360 CPUtime=0
/proc/29156/stat : 29156 (runpbsmodels) S 29154 29156 29107 0 -1 4194304 287 0 0 0 0 0 0 0 19 0 1 0 151660687 5488640 234 18446744073709551615 4194304 4889804 548682069168 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/29156/statm: 1340 234 195 169 0 51 0
[pid=29157] ppid=29156 vsize=12268 CPUtime=0
/proc/29157/stat : 29157 (pbsmodels-v1.28) S 29156 29156 29107 0 -1 4194304 425 0 0 0 0 0 0 0 18 0 1 0 151660688 12562432 383 18446744073709551615 4194304 4206940 548682068992 18446744073709551615 253316087700 0 0 4230 16384 18446744071563356171 0 0 17 0 0 0
/proc/29157/statm: 3067 383 293 3 0 167 0
[pid=29158] ppid=29157 vsize=5356 CPUtime=0.18
/proc/29158/stat : 29158 (sh) S 29157 29156 29107 0 -1 4194304 311 572 0 0 0 0 18 0 16 0 1 0 151660689 5484544 227 18446744073709551615 4194304 4889804 548682069024 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/29158/statm: 1339 227 188 169 0 50 0
[pid=29160] ppid=29158 vsize=2964 CPUtime=79.89
/proc/29160/stat : 29160 (smodels) R 29158 29156 29107 0 -1 4194304 425 0 0 0 7989 0 0 0 25 0 1 0 151660689 3035136 389 18446744073709551615 134512640 134595782 4294956656 18446744073709551615 134531217 0 0 4096 0 0 0 0 17 1 0 0
/proc/29160/statm: 741 389 212 20 0 178 0
[pid=29161] ppid=29158 vsize=12268 CPUtime=0
/proc/29161/stat : 29161 (pbout.pl) S 29158 29156 29107 0 -1 4194304 469 0 0 0 0 0 0 0 16 0 1 0 151660689 12562432 428 18446744073709551615 4194304 4206940 548682069040 18446744073709551615 253316083810 0 0 4224 0 18446744071563648864 0 0 17 0 0 0
/proc/29161/statm: 3067 428 333 3 0 167 0
Current children cumulated CPU time (s) 80.07
Current children cumulated vsize (Kb) 38216

[startup+90.054 s]
/proc/loadavg: 1.01 0.99 0.99 2/68 29161
/proc/meminfo: memFree=1615728/2055920 swapFree=4182032/4192956
[pid=29156] ppid=29154 vsize=5360 CPUtime=0
/proc/29156/stat : 29156 (runpbsmodels) S 29154 29156 29107 0 -1 4194304 287 0 0 0 0 0 0 0 19 0 1 0 151660687 5488640 234 18446744073709551615 4194304 4889804 548682069168 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/29156/statm: 1340 234 195 169 0 51 0
[pid=29157] ppid=29156 vsize=12268 CPUtime=0
/proc/29157/stat : 29157 (pbsmodels-v1.28) S 29156 29156 29107 0 -1 4194304 425 0 0 0 0 0 0 0 18 0 1 0 151660688 12562432 383 18446744073709551615 4194304 4206940 548682068992 18446744073709551615 253316087700 0 0 4230 16384 18446744071563356171 0 0 17 0 0 0
/proc/29157/statm: 3067 383 293 3 0 167 0
[pid=29158] ppid=29157 vsize=5356 CPUtime=0.18
/proc/29158/stat : 29158 (sh) S 29157 29156 29107 0 -1 4194304 311 572 0 0 0 0 18 0 16 0 1 0 151660689 5484544 227 18446744073709551615 4194304 4889804 548682069024 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/29158/statm: 1339 227 188 169 0 50 0
[pid=29160] ppid=29158 vsize=2964 CPUtime=89.9
/proc/29160/stat : 29160 (smodels) R 29158 29156 29107 0 -1 4194304 425 0 0 0 8990 0 0 0 25 0 1 0 151660689 3035136 389 18446744073709551615 134512640 134595782 4294956656 18446744073709551615 134531004 0 0 4096 0 0 0 0 17 1 0 0

################
# More data... #
################

/proc/29160/stat : 29160 (smodels) R 29158 29156 29107 0 -1 4194304 425 0 0 0 179349 1 0 0 25 0 1 0 151660689 3035136 389 18446744073709551615 134512640 134595782 4294956656 18446744073709551615 134521399 0 0 4096 0 0 0 0 17 1 0 0
/proc/29160/statm: 741 389 212 20 0 178 0
[pid=29161] ppid=29158 vsize=12268 CPUtime=0
/proc/29161/stat : 29161 (pbout.pl) S 29158 29156 29107 0 -1 4194304 469 0 0 0 0 0 0 0 16 0 1 0 151660689 12562432 428 18446744073709551615 4194304 4206940 548682069040 18446744073709551615 253316083810 0 0 4224 0 18446744071563648864 0 0 17 0 0 0
/proc/29161/statm: 3067 428 333 3 0 167 0
Current children cumulated CPU time (s) 1793.68
Current children cumulated vsize (Kb) 38216

[startup+1795.21 s]
/proc/loadavg: 1.00 1.00 1.00 2/68 29359
/proc/meminfo: memFree=1615216/2055920 swapFree=4182032/4192956
[pid=29156] ppid=29154 vsize=5360 CPUtime=0
/proc/29156/stat : 29156 (runpbsmodels) S 29154 29156 29107 0 -1 4194304 287 0 0 0 0 0 0 0 19 0 1 0 151660687 5488640 234 18446744073709551615 4194304 4889804 548682069168 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/29156/statm: 1340 234 195 169 0 51 0
[pid=29157] ppid=29156 vsize=12268 CPUtime=0
/proc/29157/stat : 29157 (pbsmodels-v1.28) S 29156 29156 29107 0 -1 4194304 425 0 0 0 0 0 0 0 18 0 1 0 151660688 12562432 383 18446744073709551615 4194304 4206940 548682068992 18446744073709551615 253316087700 0 0 4230 16384 18446744071563356171 0 0 17 0 0 0
/proc/29157/statm: 3067 383 293 3 0 167 0
[pid=29158] ppid=29157 vsize=5356 CPUtime=0.18
/proc/29158/stat : 29158 (sh) S 29157 29156 29107 0 -1 4194304 311 572 0 0 0 0 18 0 16 0 1 0 151660689 5484544 227 18446744073709551615 4194304 4889804 548682069024 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/29158/statm: 1339 227 188 169 0 50 0
[pid=29160] ppid=29158 vsize=2964 CPUtime=1794.51
/proc/29160/stat : 29160 (smodels) R 29158 29156 29107 0 -1 4194304 425 0 0 0 179450 1 0 0 25 0 1 0 151660689 3035136 389 18446744073709551615 134512640 134595782 4294956656 18446744073709551615 134521882 0 0 4096 0 0 0 0 17 1 0 0
/proc/29160/statm: 741 389 212 20 0 178 0
[pid=29161] ppid=29158 vsize=12268 CPUtime=0
/proc/29161/stat : 29161 (pbout.pl) S 29158 29156 29107 0 -1 4194304 469 0 0 0 0 0 0 0 16 0 1 0 151660689 12562432 428 18446744073709551615 4194304 4206940 548682069040 18446744073709551615 253316083810 0 0 4224 0 18446744071563648864 0 0 17 0 0 0
/proc/29161/statm: 3067 428 333 3 0 167 0
Current children cumulated CPU time (s) 1794.69
Current children cumulated vsize (Kb) 38216

[startup+1796.21 s]
/proc/loadavg: 1.00 1.00 1.00 2/68 29359
/proc/meminfo: memFree=1615216/2055920 swapFree=4182032/4192956
[pid=29156] ppid=29154 vsize=5360 CPUtime=0
/proc/29156/stat : 29156 (runpbsmodels) S 29154 29156 29107 0 -1 4194304 287 0 0 0 0 0 0 0 19 0 1 0 151660687 5488640 234 18446744073709551615 4194304 4889804 548682069168 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/29156/statm: 1340 234 195 169 0 51 0
[pid=29157] ppid=29156 vsize=12268 CPUtime=0
/proc/29157/stat : 29157 (pbsmodels-v1.28) S 29156 29156 29107 0 -1 4194304 425 0 0 0 0 0 0 0 18 0 1 0 151660688 12562432 383 18446744073709551615 4194304 4206940 548682068992 18446744073709551615 253316087700 0 0 4230 16384 18446744071563356171 0 0 17 0 0 0
/proc/29157/statm: 3067 383 293 3 0 167 0
[pid=29158] ppid=29157 vsize=5356 CPUtime=0.18
/proc/29158/stat : 29158 (sh) S 29157 29156 29107 0 -1 4194304 311 572 0 0 0 0 18 0 16 0 1 0 151660689 5484544 227 18446744073709551615 4194304 4889804 548682069024 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/29158/statm: 1339 227 188 169 0 50 0
[pid=29160] ppid=29158 vsize=2964 CPUtime=1795.51
/proc/29160/stat : 29160 (smodels) R 29158 29156 29107 0 -1 4194304 425 0 0 0 179550 1 0 0 25 0 1 0 151660689 3035136 389 18446744073709551615 134512640 134595782 4294956656 18446744073709551615 134533099 0 0 4096 0 0 0 0 17 1 0 0
/proc/29160/statm: 741 389 212 20 0 178 0
[pid=29161] ppid=29158 vsize=12268 CPUtime=0
/proc/29161/stat : 29161 (pbout.pl) S 29158 29156 29107 0 -1 4194304 469 0 0 0 0 0 0 0 16 0 1 0 151660689 12562432 428 18446744073709551615 4194304 4206940 548682069040 18446744073709551615 253316083810 0 0 4224 0 18446744071563648864 0 0 17 0 0 0
/proc/29161/statm: 3067 428 333 3 0 167 0
Current children cumulated CPU time (s) 1795.69
Current children cumulated vsize (Kb) 38216

[startup+1797.22 s]
/proc/loadavg: 1.00 1.00 1.00 2/68 29359
/proc/meminfo: memFree=1615216/2055920 swapFree=4182032/4192956
[pid=29156] ppid=29154 vsize=5360 CPUtime=0
/proc/29156/stat : 29156 (runpbsmodels) S 29154 29156 29107 0 -1 4194304 287 0 0 0 0 0 0 0 19 0 1 0 151660687 5488640 234 18446744073709551615 4194304 4889804 548682069168 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/29156/statm: 1340 234 195 169 0 51 0
[pid=29157] ppid=29156 vsize=12268 CPUtime=0
/proc/29157/stat : 29157 (pbsmodels-v1.28) S 29156 29156 29107 0 -1 4194304 425 0 0 0 0 0 0 0 18 0 1 0 151660688 12562432 383 18446744073709551615 4194304 4206940 548682068992 18446744073709551615 253316087700 0 0 4230 16384 18446744071563356171 0 0 17 0 0 0
/proc/29157/statm: 3067 383 293 3 0 167 0
[pid=29158] ppid=29157 vsize=5356 CPUtime=0.18
/proc/29158/stat : 29158 (sh) S 29157 29156 29107 0 -1 4194304 311 572 0 0 0 0 18 0 16 0 1 0 151660689 5484544 227 18446744073709551615 4194304 4889804 548682069024 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/29158/statm: 1339 227 188 169 0 50 0
[pid=29160] ppid=29158 vsize=2964 CPUtime=1796.52
/proc/29160/stat : 29160 (smodels) R 29158 29156 29107 0 -1 4194304 425 0 0 0 179651 1 0 0 25 0 1 0 151660689 3035136 389 18446744073709551615 134512640 134595782 4294956656 18446744073709551615 134531561 0 0 4096 0 0 0 0 17 1 0 0
/proc/29160/statm: 741 389 212 20 0 178 0
[pid=29161] ppid=29158 vsize=12268 CPUtime=0
/proc/29161/stat : 29161 (pbout.pl) S 29158 29156 29107 0 -1 4194304 469 0 0 0 0 0 0 0 16 0 1 0 151660689 12562432 428 18446744073709551615 4194304 4206940 548682069040 18446744073709551615 253316083810 0 0 4224 0 18446744071563648864 0 0 17 0 0 0
/proc/29161/statm: 3067 428 333 3 0 167 0
Current children cumulated CPU time (s) 1796.7
Current children cumulated vsize (Kb) 38216

[startup+1798.22 s]
/proc/loadavg: 1.00 1.00 1.00 2/68 29359
/proc/meminfo: memFree=1615216/2055920 swapFree=4182032/4192956
[pid=29156] ppid=29154 vsize=5360 CPUtime=0
/proc/29156/stat : 29156 (runpbsmodels) S 29154 29156 29107 0 -1 4194304 287 0 0 0 0 0 0 0 19 0 1 0 151660687 5488640 234 18446744073709551615 4194304 4889804 548682069168 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/29156/statm: 1340 234 195 169 0 51 0
[pid=29157] ppid=29156 vsize=12268 CPUtime=0
/proc/29157/stat : 29157 (pbsmodels-v1.28) S 29156 29156 29107 0 -1 4194304 425 0 0 0 0 0 0 0 18 0 1 0 151660688 12562432 383 18446744073709551615 4194304 4206940 548682068992 18446744073709551615 253316087700 0 0 4230 16384 18446744071563356171 0 0 17 0 0 0
/proc/29157/statm: 3067 383 293 3 0 167 0
[pid=29158] ppid=29157 vsize=5356 CPUtime=0.18
/proc/29158/stat : 29158 (sh) S 29157 29156 29107 0 -1 4194304 311 572 0 0 0 0 18 0 16 0 1 0 151660689 5484544 227 18446744073709551615 4194304 4889804 548682069024 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/29158/statm: 1339 227 188 169 0 50 0
[pid=29160] ppid=29158 vsize=2964 CPUtime=1797.52
/proc/29160/stat : 29160 (smodels) R 29158 29156 29107 0 -1 4194304 425 0 0 0 179751 1 0 0 25 0 1 0 151660689 3035136 389 18446744073709551615 134512640 134595782 4294956656 18446744073709551615 134549901 0 0 4096 0 0 0 0 17 1 0 0
/proc/29160/statm: 741 389 212 20 0 178 0
[pid=29161] ppid=29158 vsize=12268 CPUtime=0
/proc/29161/stat : 29161 (pbout.pl) S 29158 29156 29107 0 -1 4194304 469 0 0 0 0 0 0 0 16 0 1 0 151660689 12562432 428 18446744073709551615 4194304 4206940 548682069040 18446744073709551615 253316083810 0 0 4224 0 18446744071563648864 0 0 17 0 0 0
/proc/29161/statm: 3067 428 333 3 0 167 0
Current children cumulated CPU time (s) 1797.7
Current children cumulated vsize (Kb) 38216

[startup+1799.23 s]
/proc/loadavg: 1.00 1.00 1.00 2/68 29359
/proc/meminfo: memFree=1615216/2055920 swapFree=4182032/4192956
[pid=29156] ppid=29154 vsize=5360 CPUtime=0
/proc/29156/stat : 29156 (runpbsmodels) S 29154 29156 29107 0 -1 4194304 287 0 0 0 0 0 0 0 19 0 1 0 151660687 5488640 234 18446744073709551615 4194304 4889804 548682069168 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/29156/statm: 1340 234 195 169 0 51 0
[pid=29157] ppid=29156 vsize=12268 CPUtime=0
/proc/29157/stat : 29157 (pbsmodels-v1.28) S 29156 29156 29107 0 -1 4194304 425 0 0 0 0 0 0 0 18 0 1 0 151660688 12562432 383 18446744073709551615 4194304 4206940 548682068992 18446744073709551615 253316087700 0 0 4230 16384 18446744071563356171 0 0 17 0 0 0
/proc/29157/statm: 3067 383 293 3 0 167 0
[pid=29158] ppid=29157 vsize=5356 CPUtime=0.18
/proc/29158/stat : 29158 (sh) S 29157 29156 29107 0 -1 4194304 311 572 0 0 0 0 18 0 16 0 1 0 151660689 5484544 227 18446744073709551615 4194304 4889804 548682069024 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/29158/statm: 1339 227 188 169 0 50 0
[pid=29160] ppid=29158 vsize=2964 CPUtime=1798.53
/proc/29160/stat : 29160 (smodels) R 29158 29156 29107 0 -1 4194304 425 0 0 0 179852 1 0 0 25 0 1 0 151660689 3035136 389 18446744073709551615 134512640 134595782 4294956656 18446744073709551615 134521060 0 0 4096 0 0 0 0 17 1 0 0
/proc/29160/statm: 741 389 212 20 0 178 0
[pid=29161] ppid=29158 vsize=12268 CPUtime=0
/proc/29161/stat : 29161 (pbout.pl) S 29158 29156 29107 0 -1 4194304 469 0 0 0 0 0 0 0 16 0 1 0 151660689 12562432 428 18446744073709551615 4194304 4206940 548682069040 18446744073709551615 253316083810 0 0 4224 0 18446744071563648864 0 0 17 0 0 0
/proc/29161/statm: 3067 428 333 3 0 167 0
Current children cumulated CPU time (s) 1798.71
Current children cumulated vsize (Kb) 38216

[startup+1800.23 s]
/proc/loadavg: 1.00 1.00 1.00 2/68 29359
/proc/meminfo: memFree=1615216/2055920 swapFree=4182032/4192956
[pid=29156] ppid=29154 vsize=5360 CPUtime=0
/proc/29156/stat : 29156 (runpbsmodels) S 29154 29156 29107 0 -1 4194304 287 0 0 0 0 0 0 0 19 0 1 0 151660687 5488640 234 18446744073709551615 4194304 4889804 548682069168 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/29156/statm: 1340 234 195 169 0 51 0
[pid=29157] ppid=29156 vsize=12268 CPUtime=0
/proc/29157/stat : 29157 (pbsmodels-v1.28) S 29156 29156 29107 0 -1 4194304 425 0 0 0 0 0 0 0 18 0 1 0 151660688 12562432 383 18446744073709551615 4194304 4206940 548682068992 18446744073709551615 253316087700 0 0 4230 16384 18446744071563356171 0 0 17 0 0 0
/proc/29157/statm: 3067 383 293 3 0 167 0
[pid=29158] ppid=29157 vsize=5356 CPUtime=0.18
/proc/29158/stat : 29158 (sh) S 29157 29156 29107 0 -1 4194304 311 572 0 0 0 0 18 0 16 0 1 0 151660689 5484544 227 18446744073709551615 4194304 4889804 548682069024 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/29158/statm: 1339 227 188 169 0 50 0
[pid=29160] ppid=29158 vsize=2964 CPUtime=1799.53
/proc/29160/stat : 29160 (smodels) R 29158 29156 29107 0 -1 4194304 425 0 0 0 179952 1 0 0 25 0 1 0 151660689 3035136 389 18446744073709551615 134512640 134595782 4294956656 18446744073709551615 134527588 0 0 4096 0 0 0 0 17 1 0 0
/proc/29160/statm: 741 389 212 20 0 178 0
[pid=29161] ppid=29158 vsize=12268 CPUtime=0
/proc/29161/stat : 29161 (pbout.pl) S 29158 29156 29107 0 -1 4194304 469 0 0 0 0 0 0 0 16 0 1 0 151660689 12562432 428 18446744073709551615 4194304 4206940 548682069040 18446744073709551615 253316083810 0 0 4224 0 18446744071563648864 0 0 17 0 0 0
/proc/29161/statm: 3067 428 333 3 0 167 0
Current children cumulated CPU time (s) 1799.71
Current children cumulated vsize (Kb) 38216

[startup+1801.24 s]
/proc/loadavg: 1.00 1.00 1.00 2/68 29359
/proc/meminfo: memFree=1615216/2055920 swapFree=4182032/4192956
[pid=29156] ppid=29154 vsize=5360 CPUtime=0
/proc/29156/stat : 29156 (runpbsmodels) S 29154 29156 29107 0 -1 4194304 287 0 0 0 0 0 0 0 19 0 1 0 151660687 5488640 234 18446744073709551615 4194304 4889804 548682069168 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/29156/statm: 1340 234 195 169 0 51 0
[pid=29157] ppid=29156 vsize=12268 CPUtime=0
/proc/29157/stat : 29157 (pbsmodels-v1.28) S 29156 29156 29107 0 -1 4194304 425 0 0 0 0 0 0 0 18 0 1 0 151660688 12562432 383 18446744073709551615 4194304 4206940 548682068992 18446744073709551615 253316087700 0 0 4230 16384 18446744071563356171 0 0 17 0 0 0
/proc/29157/statm: 3067 383 293 3 0 167 0
[pid=29158] ppid=29157 vsize=5356 CPUtime=0.18
/proc/29158/stat : 29158 (sh) S 29157 29156 29107 0 -1 4194304 311 572 0 0 0 0 18 0 16 0 1 0 151660689 5484544 227 18446744073709551615 4194304 4889804 548682069024 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/29158/statm: 1339 227 188 169 0 50 0
[pid=29160] ppid=29158 vsize=2964 CPUtime=1800.54
/proc/29160/stat : 29160 (smodels) R 29158 29156 29107 0 -1 4194304 425 0 0 0 180053 1 0 0 25 0 1 0 151660689 3035136 389 18446744073709551615 134512640 134595782 4294956656 18446744073709551615 134518733 0 0 4096 0 0 0 0 17 1 0 0
/proc/29160/statm: 741 389 212 20 0 178 0
[pid=29161] ppid=29158 vsize=12268 CPUtime=0
/proc/29161/stat : 29161 (pbout.pl) S 29158 29156 29107 0 -1 4194304 469 0 0 0 0 0 0 0 16 0 1 0 151660689 12562432 428 18446744073709551615 4194304 4206940 548682069040 18446744073709551615 253316083810 0 0 4224 0 18446744071563648864 0 0 17 0 0 0
/proc/29161/statm: 3067 428 333 3 0 167 0
Current children cumulated CPU time (s) 1800.72
Current children cumulated vsize (Kb) 38216



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1801.24 s]
/proc/loadavg: 1.00 1.00 1.00 2/68 29359
/proc/meminfo: memFree=1615216/2055920 swapFree=4182032/4192956
[pid=29156] ppid=29154 vsize=5360 CPUtime=0
/proc/29156/stat : 29156 (runpbsmodels) S 29154 29156 29107 0 -1 4194304 287 0 0 0 0 0 0 0 19 0 1 0 151660687 5488640 234 18446744073709551615 4194304 4889804 548682069168 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/29156/statm: 1340 234 195 169 0 51 0
[pid=29157] ppid=29156 vsize=12268 CPUtime=0
/proc/29157/stat : 29157 (pbsmodels-v1.28) S 29156 29156 29107 0 -1 4194304 425 0 0 0 0 0 0 0 18 0 1 0 151660688 12562432 383 18446744073709551615 4194304 4206940 548682068992 18446744073709551615 253316087700 0 0 4230 16384 18446744071563356171 0 0 17 0 0 0
/proc/29157/statm: 3067 383 293 3 0 167 0
[pid=29158] ppid=29157 vsize=5356 CPUtime=0.18
/proc/29158/stat : 29158 (sh) S 29157 29156 29107 0 -1 4194304 311 572 0 0 0 0 18 0 16 0 1 0 151660689 5484544 227 18446744073709551615 4194304 4889804 548682069024 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/29158/statm: 1339 227 188 169 0 50 0
[pid=29160] ppid=29158 vsize=2964 CPUtime=1800.54
/proc/29160/stat : 29160 (smodels) R 29158 29156 29107 0 -1 4194304 425 0 0 0 180053 1 0 0 25 0 1 0 151660689 3035136 389 18446744073709551615 134512640 134595782 4294956656 18446744073709551615 134518543 0 0 4096 0 0 0 0 17 1 0 0
/proc/29160/statm: 741 389 212 20 0 178 0
[pid=29161] ppid=29158 vsize=12268 CPUtime=0
/proc/29161/stat : 29161 (pbout.pl) S 29158 29156 29107 0 -1 4194304 469 0 0 0 0 0 0 0 16 0 1 0 151660689 12562432 428 18446744073709551615 4194304 4206940 548682069040 18446744073709551615 253316083810 0 0 4224 0 18446744071563648864 0 0 17 0 0 0
/proc/29161/statm: 3067 428 333 3 0 167 0
Current children cumulated CPU time (s) 1800.72
Current children cumulated vsize (Kb) 38216

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Child ended because it received signal 15 (SIGTERM)

!!! problem with CPU time !!!
wait4(...,&childrusage) returns 29156 and gives childrusage.ru_utime.tv_sec=0 childrusage.ru_utime.tv_usec=999 childrusage.ru_stime.tv_sec=0 childrusage.ru_stime.tv_usec=1999
CPU time returned by wait4() is 0.002998
while last known CPU time is 1800.72

Solver probably didn't/couldn't wait for its children
Using last known CPU time as value...

Real time (s): 1801.26
CPU time (s): 1800.72
CPU user time (s): 1800.71
CPU system time (s): 0.01
CPU usage (%): 99.9698
Max. virtual memory (cumulated for all children) (Kb): 38216
The end

Launcher Data (download as text)

Begin job on node56 on Fri Jul 28 00:51:47 UTC 2006


FILE ID= 34415-1154047907

PBS_JOBID= 881714

BENCH NAME= HOME/pub/bench/PB06//final/normalized-PB06/OPT-SMALLINT/submitted-PB05/manquinho/primes-dimacs-cnf/normalized-ii8a4.opb
COMMAND LINE= ROOT/solvers/PB/PB06final/user4/pbsmodels_solver-v1.28/runpbsmodels ROOT/tmp/node56/34415-1154047907/instance-34415-1154047907.opb 511467414  ROOT/tmp/node56/34415-1154047907
RUNSOLVER COMMAND LINE= runsolver  --timestamp  -w ROOT/results/node56/watcher-34415-1154047907 -o ROOT/results/node56/solver-34415-1154047907 -C 1800 -M 1800 -S 64  ROOT/solvers/PB/PB06final/user4/pbsmodels_solver-v1.28/runpbsmodels ROOT/tmp/node56/34415-1154047907/instance-34415-1154047907.opb 511467414  ROOT/tmp/node56/34415-1154047907

MD5SUM SOLVER= f639f6b7f53e3afd14340f75c0e271ee 82c14a26114306e436bf36ef1d1c85a0 c893ceb51a2aaa4ab2f96d97b043cdf0 8c810dcfce02286b111d075d14837741 79b233270d136fcdcf9d8f80d20efef0
MD5SUM BENCH=  39250f205953aedd83d7a2bcba6bda16

RANDOM SEED= 511467414

TIMEOUT= 1800 seconds


/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.227
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5914.62
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.227
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5586.94
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:      2055920 kB
MemFree:       1617800 kB
Buffers:         26444 kB
Cached:         359820 kB
SwapCached:       4396 kB
Active:         195260 kB
Inactive:       200764 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      2055920 kB
LowFree:       1617800 kB
SwapTotal:     4192956 kB
SwapFree:      4182032 kB
Dirty:             176 kB
Writeback:           0 kB
Mapped:          14496 kB
Slab:            28068 kB
Committed_AS:   655984 kB
PageTables:       1492 kB
VmallocTotal: 536870911 kB
VmallocUsed:    264952 kB
VmallocChunk: 536605679 kB
HugePages_Total:     0
HugePages_Free:      0
Hugepagesize:     2048 kB


End job on node56 on Fri Jul 28 01:21:48 UTC 2006