Trace number 4526327

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
toulbar2 2025-06-13MSAT (TO) 3600.06 3610.9

General information on the benchmark

NamePB24/WBO/PARTIAL-LIN/wcsp/spot5/
normalized-spot5-507_wcsp.wbo
MD5SUMf81c061b693f8d6b0a85fe0e32c56d8c
Bench CategoryPARTIAL-LIN (WBO, both soft and hard constraints, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark27390
Best CPU time to get the best result obtained on this benchmark0.813106
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables884
Total number of constraints8460
Number of soft constraints8149
Number of constraints which are clauses8149
Number of constraints which are cardinality constraints (but not clauses)311
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint1
Maximum length of a constraint4
Top cost 42528
Min constraint cost 1
Max constraint cost 42528
Sum of constraints costs 333376991
Biggest number in a constraint 2
Number of bits of the biggest number in a constraint 2
Biggest sum of numbers in a constraint 5
Number of bits of the biggest sum of numbers3
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data

0.39/0.48	o 34442
0.39/0.50	o 34437
0.49/0.52	o 34422
0.49/0.54	o 34418
0.49/0.55	o 33419
0.49/0.58	o 33416
0.49/0.59	o 32416
0.49/0.59	o 31421
0.58/0.60	o 30421
0.58/0.61	o 29422
0.58/0.65	o 28423
0.58/0.65	o 28422
0.58/0.66	o 27422
0.58/0.68	o 27421
0.68/0.71	o 27420
0.68/0.74	o 27418
0.68/0.78	o 27417
0.68/0.78	o 27416
0.68/0.79	o 27414
0.68/0.79	o 27413
1.67/1.75	o 27412
1.67/1.75	o 27411
1.67/1.76	o 27410
2.77/2.81	o 27409
2.77/2.84	o 27408
4.17/4.29	o 27406
4.57/4.66	o 27401
10.55/10.67	o 27400
10.65/10.70	o 27396
170.27/170.87	o 27395
174.86/175.46	o 27394
193.01/193.69	o 27393
223.71/224.46	o 27392
223.71/224.46	o 27391
1954.97/1960.86	o 27390
3600.05/3610.90	o 27390
3600.05/3610.90	s SATISFIABLE
3600.05/3610.90	v -x1 x2 -x3 -x4 -x5 x6 -x7 -x8 -x9 -x10 x11 -x12 -x13 x14 -x15 -x16 -x17 x18 -x19 x20 -x21 x22 x23 -x24 -x25 -x26 -x27 x28 -x29 x30 -x31 -x32 -x33 -x34 -x35 x36 -x37 x38 -x39 x40 -x41 x42 -x43 x44 -x45 x46 x47 -x48 -x49 x50 -x51 x52 -x53 -x54 -x55 -x56 -x57 x58 -x59 x60 -x61 x62 x63 -x64 -x65 x66 -x67 x68 x69 -x70 -x71 x72 -x73 x74 -x75 -x76 -x77 x78 -x79 x80 -x81 -x82 x83 -x84 -x85 -x86 -x87 x88 -x89 x90 -x91 x92 -x93 -x94 x95 -x96 -x97 x98 x99 -x100 -x101 x102 -x103 x104 x105 -x106 -x107 x108 -x109 x110 -x111 x112 x113 -x114 -x115 x116 -x117 x118 -x119 x120 -x121 x122 -x123 x124 -x125 -x126 -x127 x128 -x129 x130 -x131 x132 -x133 x134 x135 -x136 -x137 x138 -x139 x140 -x141 -x142 -x143 x144 -x145 -x146 -x147 x148 -x149 x150 -x151 -x152 -x153 x154 -x155 x156 -x157 x158 -x159 x160 x161 -x162 -x163 x164 -x165 x166 -x167 x168 -x169 x170 -x171 -x172 -x173 x174 -x175 x176 x177 -x178 x179 -x180 -x181 x182 x183 -x184 -x185 x186 x187 -x188 -x189 x190 -x191 x192 -x193 x194 x195 -x196 x197 -x198 -x199 x200 -x201 -x202 -x203 -x204 -x205 x206 -x207 x208 -x209 -x210 x211 -x212 -x213 -x214 -x215 -x216 x217 -x218 -x219 x220 -x221 -x222 -x223 x224 -x225 x226 -x227 x228 -x229 -x230 x231 -x232 -x233 x234 -x235 x236 -x237 x238 -x239 -x240 -x241 x242 -x243 x244 -x245 x246 x247 -x248 -x249 -x250 -x251 -x252 -x253 x254 -x255 -x256 -x257 x258 -x259 x260 -x261 x262 -x263 x264 -x265 -x266 -x267 x268 -x269 x270 -x271 x272 x273 -x274 -x275 -x276 -x277 x278 -x279 x280 -x281 -x282 -x283 x284 -x285 -x286 x287 -x288 -x289 x290 -x291 x292 -x293 -x294 -x295 x296 -x297 x298 -x299 x300 -x301 x302 -x303 x304 -x305 x306 -x307 x308 -x309 x310 x311 -x312 -x313 x314 -x315 x316 -x317 x318 -x319 x320 x321 -x322 -x323 -x324 -x325 x326 -x327 x328 -x329 -x330 x331 -x332 -x333 -x334 -x335 x336 -x337 -x338 -x339 x340 -x341 -x342 -x343 x344 -x345 x346 -x347 x348 -x349 -x350 -x351 x352 -x353 -x354 -x355 x356 x357 -x358 -x359 x360 -x361 x362 -x363 -x364 -x365 x366 -x367 x368 -x369 -x370 x371 -x372 -x373 x374 -x375 -x376 x377 -x378 -x379 -x380 -x381 -x382 x383 -x384 -x385 x386 -x387 x388 -x389 -x390 x391 -x392 -x393 -x394 -x395 -x396 -x397 x398 -x399 x400 -x401 -x402 -x403 -x404 -x405 x406 x407 -x408 -x409 x410 -x411 -x412 -x413 x414 -x415 x416 -x417 -x418 x419 -x420 -x421 x422 -x423 x424 -x425 x426 -x427 x428 -x429 x430 -x431 -x432 -x433 x434 -x435 x436 -x437 x438 -x439 -x440 x441 -x442 x443 -x444 -x445 -x446 -x447 x448 -x449 x450 -x451 x452 -x453 x454 -x455 x456 -x457 -x458 -x459 x460 -x461 -x462 -x463 x464 -x465 x466 -x467 x468 -x469 -x470 -x471 x472 -x473 x474 -x475 x476 -x477 x478 -x479 x480 -x481 x482 x483 -x484 -x485 x486 -x487 x488 -x489 x490 -x491 x492 -x493 x494 -x495 -x496 -x497 x498 -x499 x500 -x501 x502 -x503 -x504 x505 -x506 -x507 x508 x509 -x510 -x511 -x512 -x513 x514 -x515 -x516 x517 -x518 -x519 x520 -x521 x522 -x523 -x524 -x525 x526 -x527 -x528 -x529 x530 -x531 x532 -x533 x534 -x535 x536 -x537 x538 -x539 -x540 -x541 x542 -x543 x544 x545 -x546 -x547 -x548 -x549 -x550 -x551 x552 -x553 x554 -x555 -x556 x557 -x558 -x559 -x560 -x561 x562 -x563 x564 -x565 x566 -x567 x568 -x569 -x570 -x571 x572 -x573 -x574 -x575 x576 -x577 -x578 -x579 x580 -x581 -x582 -x583 x584 -x585 -x586 -x587 x588 -x589 -x590 -x591 x592 -x593 x594 -x595 -x596 -x597 -x598 -x599 x600 -x601 x602 -x603 x604 -x605 x606 -x607 x608 x609 -x610 -x611 -x612 -x613 x614 -x615 -x616 -x617 x618 -x619 -x620 -x621 x622 -x623 -x624 -x625 x626 -x627 x628 -x629 x630 -x631 x632 -x633 x634 -x635 -x636 -x637 x638 -x639 -x640 -x641 x642 -x643 -x644 -x645 x646 -x647 -x648 x649 -x650 -x651 -x652 -x653 x654 x655 -x656 -x657 -x658 -x659 x660 -x661 x662 -x663 -x664 -x665 -x666 -x667 x668 -x669 x670 -x671 -x672 -x673 x674 -x675 -x676 -x677 x678 -x679 -x680 -x681 x682 -x683 x684 -x685 x686 -x687 -x688 -x689 x690 -x691 -x692 -x693 x694 -x695 -x696 -x697 x698 -x699 -x700 x701 -x702 x703 -x704 -x705 -x706 -x707 -x708 -x709 x710 -x711 -x712 -x713 x714 -x715 -x716 -x717 x718 -x719 -x720 -x721 x722 -x723 x724 -x725 x726 -x727 -x728 x729 -x730 x731 -x732 -x733 -x734 -x735 x736 -x737 -x738 -x739 x740 -x741 x742 -x743 x744 -x745 -x746 -x747 x748 -x749 x750 -x751 -x752 -x753 x754 -x755 -x756 -x757 x758 -x759 -x760 x761 -x762 -x763 -x764 -x765 x766 x767 -x768 -x769 -x770 -x771 x772 -x773 -x774 -x775 x776 -x777 x778 -x779 x780 -x781 x782 -x783 x784 -x785 x786 -x787 x788 -x789 x790 -x791 x792 -x793 x794 -x795 -x796 -x797 x798 -x799 -x800 -x801 x802 -x803 x804 -x805 -x806 -x807 x808 -x809 x810 -x811 -x812 -x813 x814 -x815 x816 -x817 -x818 -x819 x820 -x821 -x822 x823 -x824 -x825 x826 -x827 x828 -x829 -x830 x831 -x832 -x833 -x834 -x835 -x836 -x837 x838 -x839 -x840 -x841 x842 -x843 -x844 -x845 x846 -x847 -x848 x849 -x850 x851 -x852 -x853 -x854 -x855 x856 -x857 -x858 -x859 -x860 x861 -x862 -x863 -x864 x865 -x866 x867 -x868 -x869 -x870 -x871 -x872 x873 -x874 -x875 x876 -x877 -x878 x879 -x880 -x881 x882 x883 -x884

Verifier Data

OK	27390

Watcher Data

runsolver version 3.4.2 (svn: 4680) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 63665 MiB (61274 MiB free)
  memory of node 1: 64507 MiB (55937 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4526327-1751629878/watcher-4526327-1751629878 -o /tmp/evaluation-result-4526327-1751629878/solver-4526327-1751629878 -C 3600 -W 3700 -M 31000 --bin-var /tmp/runsolver-exch-nodeC022-1751626267-89423 --watchdog 3760 toulbar2.sh toulbar2 HOME/instance-4526327-1751629878.wbo -timer=3600 -hve=512 -f=3 -n: -precision=0 

running on 2 cores: 1,3

CPU governors:
  performance: 1 3

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 3600 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 3630 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 3700 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Linux 5.14.0-503.14.1.el9_5.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=89934, runsolver pid=89931

[startup+0.100063 s]*
/proc/loadavg: 4.12 4.09 4.09 5/267 89936
/proc/meminfo: memFree=119977192/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=58956 memory=49024 CPUtime=0.08 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 11483 0 0 0 7 1 0 0 20 0 1 0 78665581 60370944 12256 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 14739 12256 928 1162 0 13348 0
Current children cumulated CPU time: 0.08 s
Current children cumulated vsize: 63148 KiB
Current children cumulated memory: 52096 KiB

[startup+0.205694 s]*
/proc/loadavg: 4.12 4.09 4.09 5/267 89936
/proc/meminfo: memFree=119953000/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=84436 memory=74228 CPUtime=0.19 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 17919 0 0 0 17 2 0 0 20 0 1 0 78665581 86462464 18557 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 21109 18557 1024 1162 0 19718 0
Current children cumulated CPU time: 0.19 s
Current children cumulated vsize: 88628 KiB
Current children cumulated memory: 77300 KiB

[startup+0.305693 s]*
/proc/loadavg: 4.12 4.09 4.09 5/267 89936
/proc/meminfo: memFree=119952244/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=88136 memory=77940 CPUtime=0.3 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 18858 0 0 0 27 3 0 0 20 0 1 0 78665581 90251264 19485 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 22034 19485 1024 1162 0 20643 0
Current children cumulated CPU time: 0.3 s
Current children cumulated vsize: 92328 KiB
Current children cumulated memory: 81012 KiB

[startup+0.700212 s]
/proc/loadavg: 4.12 4.09 4.09 5/267 89936
/proc/meminfo: memFree=119937124/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=98128 memory=89076 CPUtime=0.68 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 21479 0 0 0 65 3 0 0 20 0 1 0 78665581 100483072 22269 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 24532 22269 1184 1162 0 23141 0
Current children cumulated CPU time: 0.68 s
Current children cumulated vsize: 102320 KiB
Current children cumulated memory: 92148 KiB

[startup+1.50017 s]
/proc/loadavg: 4.12 4.09 4.09 5/265 89950
/proc/meminfo: memFree=120930280/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=98612 memory=89460 CPUtime=1.47 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 21577 0 0 0 144 3 0 0 20 0 1 0 78665581 100978688 22365 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 24653 22365 1184 1162 0 23262 0
Current children cumulated CPU time: 1.47 s
Current children cumulated vsize: 102804 KiB
Current children cumulated memory: 92532 KiB

[startup+3.10539 s]
/proc/loadavg: 4.11 4.09 4.09 5/267 89982
/proc/meminfo: memFree=120657244/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=99008 memory=89716 CPUtime=3.07 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 21647 0 0 0 304 3 0 0 20 0 1 0 78665581 101384192 22429 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 24752 22429 1184 1162 0 23361 0
Current children cumulated CPU time: 3.07 s
Current children cumulated vsize: 103200 KiB
Current children cumulated memory: 92788 KiB

[startup+6.30019 s]
/proc/loadavg: 4.11 4.09 4.09 5/267 89982
/proc/meminfo: memFree=120641480/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=99584 memory=90228 CPUtime=6.26 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 21769 0 0 0 623 3 0 0 20 0 1 0 78665581 101974016 22557 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 24896 22557 1184 1162 0 23505 0
Current children cumulated CPU time: 6.26 s
Current children cumulated vsize: 103776 KiB
Current children cumulated memory: 93300 KiB

[startup+12.7002 s]
/proc/loadavg: 4.09 4.09 4.09 5/267 89982
/proc/meminfo: memFree=120649228/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=100740 memory=91124 CPUtime=12.65 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 22017 0 0 0 1261 4 0 0 20 0 1 0 78665581 103157760 22781 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 25185 22781 1184 1162 0 23794 0
Current children cumulated CPU time: 12.65 s
Current children cumulated vsize: 104932 KiB
Current children cumulated memory: 94196 KiB

[startup+25.5002 s]
/proc/loadavg: 4.08 4.08 4.09 5/267 89982
/proc/meminfo: memFree=120645700/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=103048 memory=92996 CPUtime=25.4 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 22681 0 0 0 2536 4 0 0 20 0 1 0 78665581 105521152 23249 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 25762 23249 1184 1162 0 24371 0
Current children cumulated CPU time: 25.4 s
Current children cumulated vsize: 107240 KiB
Current children cumulated memory: 96068 KiB

[startup+51.1055 s]
/proc/loadavg: 4.12 4.09 4.09 5/266 89983
/proc/meminfo: memFree=120647624/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=106120 memory=95644 CPUtime=50.93 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 23574 0 0 0 5088 5 0 0 20 0 1 0 78665581 108666880 23911 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 26530 23911 1184 1162 0 25139 0
Current children cumulated CPU time: 50.93 s
Current children cumulated vsize: 110312 KiB
Current children cumulated memory: 98716 KiB

[startup+102.3 s]
/proc/loadavg: 4.11 4.09 4.09 5/267 89987
/proc/meminfo: memFree=120633128/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=109192 memory=97844 CPUtime=101.98 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 23887 0 0 0 10192 6 0 0 20 0 1 0 78665581 111812608 24461 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 27298 24461 1184 1162 0 25907 0
Current children cumulated CPU time: 101.98 s
Current children cumulated vsize: 113384 KiB
Current children cumulated memory: 100916 KiB

[startup+162.3 s]
/proc/loadavg: 4.13 4.10 4.09 5/267 89989
/proc/meminfo: memFree=120643068/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=112264 memory=101960 CPUtime=161.8 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 24659 0 0 0 16174 6 0 0 20 0 1 0 78665581 114958336 25490 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 28066 25490 1184 1162 0 26675 0
Current children cumulated CPU time: 161.8 s
Current children cumulated vsize: 116456 KiB
Current children cumulated memory: 105032 KiB

[startup+222.3 s]
/proc/loadavg: 4.49 4.24 4.14 5/267 90037
/proc/meminfo: memFree=121422036/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=118408 memory=104548 CPUtime=221.62 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 24805 0 0 0 22156 6 0 0 20 0 1 0 78665581 121249792 26137 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 29602 26137 1184 1162 0 28211 0
Current children cumulated CPU time: 221.62 s
Current children cumulated vsize: 122600 KiB

################
# More data... #
################

/proc/loadavg: 4.13 4.10 4.09 5/268 90229
/proc/meminfo: memFree=111800668/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=394900 memory=280440 CPUtime=3092.98 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 28803 0 0 0 309281 17 0 0 20 0 1 0 78665581 404377600 70110 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 98725 70110 1184 1162 0 97334 0
Current children cumulated CPU time: 3092.98 s
Current children cumulated vsize: 399092 KiB
Current children cumulated memory: 283512 KiB

[startup+3162.3 s]
/proc/loadavg: 3.83 4.03 4.07 4/260 90261
/proc/meminfo: memFree=121163952/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=394900 memory=282500 CPUtime=3152.8 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 28809 0 0 0 315263 17 0 0 20 0 1 0 78665581 404377600 70625 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 98725 70625 1184 1162 0 97334 0
Current children cumulated CPU time: 3152.8 s
Current children cumulated vsize: 399092 KiB
Current children cumulated memory: 285572 KiB

[startup+3222.3 s]
/proc/loadavg: 3.35 3.85 4.00 4/260 90263
/proc/meminfo: memFree=121133356/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=399032 memory=288672 CPUtime=3212.61 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 29388 0 0 0 321244 17 0 0 20 0 1 0 78665581 408608768 72168 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 99758 72168 1184 1162 0 98367 0
Current children cumulated CPU time: 3212.61 s
Current children cumulated vsize: 403224 KiB
Current children cumulated memory: 291744 KiB

[startup+3282.3 s]
/proc/loadavg: 3.20 3.71 3.94 4/261 90266
/proc/meminfo: memFree=121129388/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=399032 memory=292280 CPUtime=3272.43 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 29414 0 0 0 327226 17 0 0 20 0 1 0 78665581 408608768 73070 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 99758 73070 1184 1162 0 98367 0
Current children cumulated CPU time: 3272.43 s
Current children cumulated vsize: 403224 KiB
Current children cumulated memory: 295352 KiB

[startup+3342.3 s]
/proc/loadavg: 3.12 3.59 3.89 4/260 90268
/proc/meminfo: memFree=121126696/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=399032 memory=296376 CPUtime=3332.25 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 29418 0 0 0 333208 17 0 0 20 0 1 0 78665581 408608768 74094 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 99758 74094 1184 1162 0 98367 0
Current children cumulated CPU time: 3332.25 s
Current children cumulated vsize: 403224 KiB
Current children cumulated memory: 299448 KiB

[startup+3402.3 s]
/proc/loadavg: 3.08 3.50 3.84 4/260 90270
/proc/meminfo: memFree=121114056/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=399032 memory=298424 CPUtime=3392.07 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 29499 0 0 0 339190 17 0 0 20 0 1 0 78665581 408608768 74606 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 99758 74606 1184 1162 0 98367 0
Current children cumulated CPU time: 3392.07 s
Current children cumulated vsize: 403224 KiB
Current children cumulated memory: 301496 KiB

[startup+3462.3 s]
/proc/loadavg: 3.06 3.42 3.79 4/260 90276
/proc/meminfo: memFree=121108636/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=399032 memory=300472 CPUtime=3451.89 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 29506 0 0 0 345172 17 0 0 20 0 1 0 78665581 408608768 75118 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 99758 75118 1184 1162 0 98367 0
Current children cumulated CPU time: 3451.89 s
Current children cumulated vsize: 403224 KiB
Current children cumulated memory: 303544 KiB

[startup+3522.3 s]
/proc/loadavg: 3.07 3.36 3.74 4/260 90278
/proc/meminfo: memFree=121104636/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=399032 memory=302520 CPUtime=3511.71 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 29509 0 0 0 351154 17 0 0 20 0 1 0 78665581 408608768 75630 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 99758 75630 1184 1162 0 98367 0
Current children cumulated CPU time: 3511.71 s
Current children cumulated vsize: 403224 KiB
Current children cumulated memory: 305592 KiB

[startup+3582.3 s]
/proc/loadavg: 3.13 3.32 3.70 4/260 90282
/proc/meminfo: memFree=121104844/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=399032 memory=304568 CPUtime=3571.54 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 29597 0 0 0 357136 18 0 0 20 0 1 0 78665581 408608768 76142 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 99758 76142 1184 1162 0 98367 0
Current children cumulated CPU time: 3571.54 s
Current children cumulated vsize: 403224 KiB
Current children cumulated memory: 307640 KiB



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+3610.9 s]
/proc/loadavg: 3.08 3.29 3.68 4/260 90283
/proc/meminfo: memFree=121100756/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=399032 memory=308664 CPUtime=3600.05 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 29602 0 0 0 359987 18 0 0 20 0 1 0 78665581 408608768 77166 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 99758 77166 1184 1162 0 98367 0
Current children cumulated CPU time: 3600.05 s
Current children cumulated vsize: 403224 KiB
Current children cumulated memory: 311736 KiB

Sending SIGTERM to process tree (bottom up)

[startup+3610.9 s]
# the end of solver process 89934 was just reported to runsolver
# Child status: 0

# cumulated CPU time of all completed processes:  user=3599.88 s, system=0.18592 s

Solver just ended.
Sleeping 2 seconds
??? end of watcher thread
Dumping a history of the last processes samples

[startup+3610.9 s]
/proc/loadavg: 3.08 3.29 3.68 4/260 90283
/proc/meminfo: memFree=121100756/131249996 swapFree=33010416/33010684
[pid=89934] ppid=89931 vsize=4192 memory=3072 CPUtime=0 cores=1,3
/proc/89934/stat : 89934 (toulbar2.sh) S 89931 89934 89411 0 -1 4194304 206 0 0 0 0 0 0 0 20 0 1 0 78665580 4292608 768 18446744073709551615 94913043058688 94913043953221 140720837457312 0 0 0 65536 4 65538 1 0 0 17 3 0 0 0 0 0 94913044192272 94913044240016 94913061007360 140720837462325 140720837462501 140720837462501 140720837468105 0
/proc/89934/statm: 1048 768 704 219 0 110 0
[pid=89936] ppid=89934 vsize=399032 memory=308664 CPUtime=3600.05 cores=1,3
/proc/89936/stat : 89936 (toulbar2) R 89934 89934 89411 0 -1 4194304 29602 0 0 0 359987 18 0 0 20 0 1 0 78665581 408608768 77166 18446744073709551615 4198400 8956741 140725960374896 0 0 0 0 0 33570818 0 0 0 17 1 0 0 0 0 0 9785056 9890480 10199040 140725960378739 140725960378858 140725960378858 140725960384460 0
/proc/89936/statm: 99758 77166 1184 1162 0 98367 0
Current children cumulated CPU time: 3600.05 s
Current children cumulated vsize: 403224 KiB
Current children cumulated memory: 311736 KiB

Child status: 0

Real time (s): 3610.9
CPU time (s): 3600.06
CPU user time (s): 3599.88
CPU system time (s): 0.18592
CPU usage (%): 99.6998
Max. virtual memory (cumulated for all children) (KiB): 403224
Max. memory (cumulated for all children) (KiB): 311736

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 3599.88
system time used= 0.18592
maximum resident set size= 356112
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 29816
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 93
involuntary context switches= 1954


# summary of solver processes directly reported to runsolver:
#   pid: 89934
#   total CPU time (s): 3600.06
#   total CPU user time (s): 3599.88
#   total CPU system time (s): 0.18592

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 6.76574 second user time and 15.7829 second system time

The end

Launcher Data

Begin job on nodeC022 at 2025-07-04 13:51:18
IDJOB=4526327
IDBENCH=144038
IDSOLVER=3383
FILE ID=nodeC022/4526327-1751629878
RUNJOBID= nodeC022-1751626267-89423
SLURM_JOB_ID= 9598954
Free space on /tmp= 432364 MiB

SOLVER NAME= toulbar2 2025-06-13
BENCH NAME= PB24/WBO/PARTIAL-LIN/wcsp/spot5/normalized-spot5-507_wcsp.wbo
COMMAND LINE= toulbar2.sh toulbar2 BENCHNAME -timer=TIMELIMIT -hve=512 -f=3 -n: -precision=0
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4526327-1751629878/watcher-4526327-1751629878 -o /tmp/evaluation-result-4526327-1751629878/solver-4526327-1751629878 -C 3600 -W 3700 -M 31000 --bin-var /tmp/runsolver-exch-nodeC022-1751626267-89423 --watchdog 3760  toulbar2.sh toulbar2 HOME/instance-4526327-1751629878.wbo -timer=3600 -hve=512 -f=3 -n: -precision=0

TIME LIMIT= 3600 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 2

MD5SUM BENCH= f81c061b693f8d6b0a85fe0e32c56d8c
RANDOM SEED=1769204143

nodeC022 Linux 5.14.0-503.14.1.el9_5.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 15 12:04:32 UTC 2024

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3599.689
cache size	: 15360 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 pti ibrs ibpb stibp tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts vnmi
vmx flags	: vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple shadow_vmcs pml ept_violation_ve
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit mmio_stale_data
bogomips	: 6999.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3599.551
cache size	: 15360 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 pti ibrs ibpb stibp tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts vnmi
vmx flags	: vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple shadow_vmcs pml ept_violation_ve
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit mmio_stale_data
bogomips	: 6999.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3600.850
cache size	: 15360 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 pti ibrs ibpb stibp tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts vnmi
vmx flags	: vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple shadow_vmcs pml ept_violation_ve
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit mmio_stale_data
bogomips	: 6999.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3599.799
cache size	: 15360 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 pti ibrs ibpb stibp tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts vnmi
vmx flags	: vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple shadow_vmcs pml ept_violation_ve
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit mmio_stale_data
bogomips	: 6999.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3599.353
cache size	: 15360 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 pti ibrs ibpb stibp tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts vnmi
vmx flags	: vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple shadow_vmcs pml ept_violation_ve
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit mmio_stale_data
bogomips	: 6999.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3599.250
cache size	: 15360 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 pti ibrs ibpb stibp tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts vnmi
vmx flags	: vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple shadow_vmcs pml ept_violation_ve
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit mmio_stale_data
bogomips	: 6999.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3599.921
cache size	: 15360 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 pti ibrs ibpb stibp tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts vnmi
vmx flags	: vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple shadow_vmcs pml ept_violation_ve
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit mmio_stale_data
bogomips	: 6999.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3603.029
cache size	: 15360 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 pti ibrs ibpb stibp tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts vnmi
vmx flags	: vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple shadow_vmcs pml ept_violation_ve
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit mmio_stale_data
bogomips	: 6999.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       131249996 kB
MemFree:        120025520 kB
MemAvailable:   127636832 kB
Buffers:            2128 kB
Cached:          8362184 kB
SwapCached:           32 kB
Active:          2780536 kB
Inactive:        7711212 kB
Active(anon):    2141288 kB
Inactive(anon):     4176 kB
Active(file):     639248 kB
Inactive(file):  7707036 kB
Unevictable:        3072 kB
Mlocked:               0 kB
SwapTotal:      33010684 kB
SwapFree:       33010416 kB
Zswap:                 0 kB
Zswapped:              0 kB
Dirty:              6768 kB
Writeback:             0 kB
AnonPages:       2130532 kB
Mapped:           118204 kB
Shmem:             18028 kB
KReclaimable:     208132 kB
Slab:             299804 kB
SReclaimable:     208132 kB
SUnreclaim:        91672 kB
KernelStack:        4320 kB
PageTables:         7312 kB
SecPageTables:         0 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    98635680 kB
Committed_AS:    2523580 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      150568 kB
VmallocChunk:          0 kB
Percpu:             3776 kB
HardwareCorrupted:     0 kB
AnonHugePages:   1974272 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
FileHugePages:         0 kB
FilePmdMapped:         0 kB
CmaTotal:              0 kB
CmaFree:               0 kB
Unaccepted:            0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB
DirectMap4k:      205524 kB
DirectMap2M:     7954432 kB
DirectMap1G:    127926272 kB

Free space on /tmp at the end= 432368 MiB
End job on nodeC022 at 2025-07-04 14:51:29