Trace number 4526354

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
toulbar2 2025-06-13MSAT 3599.7 3609.47

General information on the benchmark

NamePB24/WBO/PARTIAL-LIN/wcsp/spot5/
normalized-spot5-412_wcsp.wbo
MD5SUM84bec45daf3e2f2cbd3688999a3e29ea
Bench CategoryPARTIAL-LIN (WBO, both soft and hard constraints, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark32381
Best CPU time to get the best result obtained on this benchmark0.763593
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables844
Total number of constraints7574
Number of soft constraints7274
Number of constraints which are clauses7274
Number of constraints which are cardinality constraints (but not clauses)300
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint1
Maximum length of a constraint4
Top cost 48484
Min constraint cost 1
Max constraint cost 48484
Sum of constraints costs 338175899
Biggest number in a constraint 2
Number of bits of the biggest number in a constraint 2
Biggest sum of numbers in a constraint 5
Number of bits of the biggest sum of numbers3
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.20/0.25	o 43398
0.20/0.26	o 41403
0.20/0.27	o 41399
0.30/0.31	o 41397
0.30/0.32	o 41395
0.30/0.33	o 40402
0.30/0.34	o 40401
0.30/0.35	o 40400
0.30/0.37	o 39391
0.30/0.40	o 38398
0.39/0.41	o 38397
0.39/0.42	o 37398
0.39/0.43	o 37395
0.39/0.47	o 36398
0.39/0.49	o 36394
0.49/0.56	o 35398
0.68/0.72	o 34402
0.78/0.83	o 34400
0.78/0.86	o 33399
0.78/0.87	o 33398
1.18/1.26	o 33397
1.28/1.33	o 33394
1.28/1.33	o 33393
1.28/1.33	o 33390
1.28/1.33	o 33389
1.28/1.35	o 33385
2.48/2.59	o 32391
2.48/2.59	o 32390
21.61/21.78	o 32389
47.74/47.94	o 32388
47.74/47.95	o 32387
48.54/48.75	o 32386
50.13/50.37	o 32385
50.13/50.38	o 32384
1828.30/1833.33	o 32383
3599.61/3609.47	o 32383
3599.61/3609.47	s SATISFIABLE
3599.61/3609.47	v -x1 x2 -x3 -x4 -x5 x6 -x7 -x8 -x9 -x10 x11 -x12 -x13 x14 -x15 -x16 -x17 x18 -x19 x20 -x21 x22 x23 -x24 -x25 -x26 -x27 x28 -x29 -x30 -x31 x32 -x33 x34 -x35 -x36 -x37 x38 -x39 x40 -x41 x42 -x43 x44 -x45 x46 x47 -x48 -x49 x50 -x51 -x52 -x53 x54 -x55 x56 -x57 -x58 -x59 x60 -x61 x62 x63 -x64 -x65 x66 -x67 x68 x69 -x70 -x71 x72 -x73 x74 -x75 x76 -x77 -x78 -x79 -x80 -x81 x82 x83 -x84 -x85 -x86 -x87 x88 -x89 x90 -x91 -x92 -x93 x94 x95 -x96 -x97 x98 -x99 x100 x101 -x102 -x103 x104 -x105 x106 x107 -x108 -x109 x110 -x111 x112 x113 -x114 -x115 x116 -x117 x118 -x119 x120 -x121 x122 -x123 x124 -x125 -x126 -x127 x128 -x129 x130 x131 -x132 -x133 x134 -x135 x136 -x137 x138 -x139 x140 -x141 -x142 -x143 x144 -x145 -x146 -x147 x148 -x149 x150 -x151 -x152 -x153 x154 -x155 x156 x157 -x158 -x159 x160 -x161 x162 -x163 x164 -x165 x166 -x167 x168 -x169 x170 -x171 -x172 -x173 x174 -x175 x176 x177 -x178 x179 -x180 -x181 x182 x183 -x184 -x185 x186 x187 -x188 -x189 x190 -x191 x192 -x193 x194 x195 -x196 x197 -x198 -x199 -x200 -x201 x202 -x203 -x204 -x205 x206 -x207 -x208 -x209 x210 x211 -x212 -x213 -x214 -x215 x216 -x217 -x218 -x219 x220 -x221 -x222 -x223 x224 -x225 x226 -x227 x228 -x229 x230 -x231 -x232 -x233 x234 -x235 x236 -x237 -x238 x239 -x240 -x241 x242 -x243 x244 -x245 x246 x247 -x248 -x249 -x250 -x251 -x252 -x253 x254 -x255 -x256 -x257 x258 -x259 x260 -x261 x262 -x263 -x264 x265 -x266 -x267 x268 -x269 x270 -x271 x272 x273 -x274 -x275 -x276 -x277 x278 -x279 x280 -x281 -x282 -x283 x284 -x285 -x286 -x287 x288 -x289 x290 -x291 x292 -x293 -x294 x295 -x296 -x297 x298 -x299 x300 -x301 x302 -x303 x304 x305 -x306 -x307 -x308 -x309 x310 -x311 x312 -x313 x314 x315 -x316 -x317 x318 -x319 x320 -x321 x322 -x323 x324 -x325 -x326 x327 -x328 -x329 x330 x331 -x332 -x333 -x334 x335 -x336 -x337 x338 -x339 x340 -x341 -x342 x343 -x344 -x345 x346 -x347 -x348 -x349 x350 -x351 -x352 -x353 x354 -x355 -x356 x357 -x358 x359 -x360 -x361 -x362 -x363 x364 -x365 x366 -x367 -x368 -x369 x370 -x371 -x372 -x373 x374 -x375 x376 -x377 x378 -x379 x380 -x381 -x382 x383 -x384 -x385 x386 -x387 -x388 x389 -x390 x391 -x392 -x393 -x394 -x395 -x396 -x397 x398 -x399 -x400 x401 -x402 -x403 x404 x405 -x406 -x407 -x408 -x409 x410 -x411 -x412 -x413 -x414 -x415 x416 -x417 -x418 -x419 x420 -x421 x422 -x423 -x424 -x425 x426 x427 -x428 -x429 -x430 -x431 x432 -x433 x434 -x435 -x436 -x437 x438 x439 -x440 -x441 x442 -x443 x444 -x445 x446 -x447 x448 -x449 -x450 -x451 x452 x453 -x454 -x455 x456 -x457 -x458 -x459 x460 -x461 -x462 -x463 x464 -x465 x466 -x467 x468 -x469 x470 -x471 x472 -x473 x474 -x475 -x476 -x477 x478 -x479 -x480 -x481 x482 -x483 x484 -x485 x486 -x487 -x488 -x489 x490 -x491 x492 -x493 x494 x495 -x496 -x497 x498 -x499 x500 -x501 x502 -x503 x504 -x505 x506 -x507 x508 -x509 x510 x511 -x512 -x513 -x514 -x515 x516 -x517 x518 -x519 x520 -x521 -x522 -x523 x524 -x525 x526 x527 -x528 -x529 -x530 -x531 x532 -x533 x534 -x535 -x536 -x537 x538 -x539 -x540 -x541 x542 -x543 x544 -x545 -x546 x547 -x548 -x549 x550 -x551 x552 -x553 x554 -x555 x556 -x557 -x558 -x559 x560 -x561 x562 x563 -x564 -x565 -x566 -x567 -x568 -x569 x570 -x571 x572 -x573 x574 -x575 -x576 -x577 -x578 -x579 x580 -x581 x582 -x583 x584 -x585 -x586 -x587 x588 -x589 x590 -x591 -x592 -x593 x594 -x595 -x596 -x597 x598 -x599 -x600 -x601 x602 -x603 -x604 -x605 x606 -x607 x608 -x609 x610 -x611 x612 -x613 -x614 x615 -x616 -x617 x618 x619 -x620 -x621 -x622 -x623 x624 -x625 x626 -x627 -x628 -x629 x630 -x631 x632 -x633 -x634 -x635 x636 -x637 -x638 -x639 x640 -x641 x642 -x643 -x644 -x645 x646 -x647 -x648 -x649 x650 -x651 -x652 -x653 x654 -x655 x656 -x657 x658 -x659 x660 -x661 x662 -x663 x664 -x665 -x666 x667 -x668 -x669 x670 x671 -x672 -x673 -x674 -x675 x676 -x677 -x678 -x679 x680 -x681 -x682 -x683 x684 -x685 -x686 -x687 x688 -x689 -x690 -x691 x692 -x693 x694 -x695 x696 -x697 x698 -x699 x700 -x701 -x702 -x703 x704 -x705 -x706 -x707 x708 -x709 -x710 -x711 x712 -x713 x714 -x715 -x716 -x717 -x718 -x719 x720 -x721 -x722 x723 -x724 -x725 x726 -x727 -x728 -x729 x730 x731 -x732 -x733 -x734 -x735 x736 -x737 x738 -x739 -x740 -x741 x742 -x743 -x744 -x745 x746 -x747 -x748 -x749 x750 -x751 x752 -x753 x754 -x755 -x756 -x757 x758 -x759 -x760 -x761 x762 -x763 -x764 -x765 x766 -x767 -x768 -x769 x770 -x771 -x772 -x773 x774 -x775 x776 -x777 -x778 x779 -x780 -x781 x782 -x783 -x784 -x785 -x786 -x787 x788 -x789 -x790 -x791 x792 -x793 -x794 -x795 x796 -x797 -x798 -x799 x800 -x801 x802 -x803 x804 -x805 x806 -x807 x808 -x809 -x810 -x811 x812 -x813 -x814 -x815 x816 -x817 x818 -x819 -x820 -x821 x822 -x823 x824 -x825 -x826 -x827 x828 -x829 -x830 -x831 x832 -x833 -x834 x835 -x836 -x837 x838 -x839 -x840 x841 -x842 -x843 -x844

Verifier Data 

OK	32383

Watcher Data 

runsolver version 3.4.2 (svn: 4680) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 63665 MiB (62144 MiB free)
  memory of node 1: 64507 MiB (56519 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4526354-1751630072/watcher-4526354-1751630072 -o /tmp/evaluation-result-4526354-1751630072/solver-4526354-1751630072 -C 3600 -W 3700 -M 31000 --bin-var /tmp/runsolver-exch-nodeC022-1751626267-89421 --watchdog 3760 toulbar2.sh toulbar2 HOME/instance-4526354-1751630072.wbo -timer=3600 -hve=512 -f=3 -n: -precision=0 

running on 2 cores: 0,2

CPU governors:
  performance: 0 2

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 3600 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 3630 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 3700 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Linux 5.14.0-503.14.1.el9_5.x86_64
Current StackSize limit: 8192 KiB

solver pid=90035, runsolver pid=90032

[startup+0.100066 s]*
/proc/loadavg: 4.71 4.25 4.14 5/267 90037
/proc/meminfo: memFree=121451884/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=70116 memory=60672 CPUtime=0.09 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 14325 0 0 0 8 1 0 0 20 0 1 0 78684922 71798784 15168 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 17529 15168 992 1162 0 16138 0
Current children cumulated CPU time: 0.09 s
Current children cumulated vsize: 74308 KiB
Current children cumulated memory: 63616 KiB

[startup+0.205719 s]*
/proc/loadavg: 4.71 4.25 4.14 5/267 90037
/proc/meminfo: memFree=121444576/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=78708 memory=68840 CPUtime=0.2 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 16543 0 0 0 18 2 0 0 20 0 1 0 78684922 80596992 17210 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 19677 17210 1024 1162 0 18286 0
Current children cumulated CPU time: 0.2 s
Current children cumulated vsize: 82900 KiB
Current children cumulated memory: 71784 KiB

[startup+0.305504 s]*
/proc/loadavg: 4.71 4.25 4.14 5/267 90037
/proc/meminfo: memFree=121440544/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=81684 memory=72680 CPUtime=0.3 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 17375 0 0 0 28 2 0 0 20 0 1 0 78684922 83644416 18170 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 20421 18170 1184 1162 0 19030 0
Current children cumulated CPU time: 0.3 s
Current children cumulated vsize: 85876 KiB
Current children cumulated memory: 75624 KiB

[startup+0.700156 s]
/proc/loadavg: 4.71 4.25 4.14 5/267 90037
/proc/meminfo: memFree=121440544/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=82660 memory=73448 CPUtime=0.68 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 17570 0 0 0 66 2 0 0 20 0 1 0 78684922 84643840 18362 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 20665 18362 1184 1162 0 19274 0
Current children cumulated CPU time: 0.68 s
Current children cumulated vsize: 86852 KiB
Current children cumulated memory: 76392 KiB

[startup+1.50016 s]
/proc/loadavg: 4.71 4.25 4.14 5/267 90037
/proc/meminfo: memFree=121440812/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=83400 memory=74088 CPUtime=1.48 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 17727 0 0 0 146 2 0 0 20 0 1 0 78684922 85401600 18522 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 20850 18522 1184 1162 0 19459 0
Current children cumulated CPU time: 1.48 s
Current children cumulated vsize: 87592 KiB
Current children cumulated memory: 77032 KiB

[startup+3.10537 s]
/proc/loadavg: 4.71 4.25 4.14 5/267 90037
/proc/meminfo: memFree=121436748/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=83784 memory=74472 CPUtime=3.08 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 17820 0 0 0 306 2 0 0 20 0 1 0 78684922 85794816 18618 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 20946 18618 1184 1162 0 19555 0
Current children cumulated CPU time: 3.08 s
Current children cumulated vsize: 87976 KiB
Current children cumulated memory: 77416 KiB

[startup+6.30015 s]
/proc/loadavg: 4.65 4.25 4.14 5/267 90037
/proc/meminfo: memFree=121428936/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=85328 memory=75880 CPUtime=6.26 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 18156 0 0 0 624 2 0 0 20 0 1 0 78684922 87375872 18970 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 21332 18970 1184 1162 0 19941 0
Current children cumulated CPU time: 6.26 s
Current children cumulated vsize: 89520 KiB
Current children cumulated memory: 78824 KiB

[startup+12.7001 s]
/proc/loadavg: 4.60 4.24 4.14 5/267 90037
/proc/meminfo: memFree=121425108/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=86864 memory=76468 CPUtime=12.64 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 18697 0 0 0 1262 2 0 0 20 0 1 0 78684922 88948736 19117 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 21716 19117 1184 1162 0 20325 0
Current children cumulated CPU time: 12.64 s
Current children cumulated vsize: 91056 KiB
Current children cumulated memory: 79412 KiB

[startup+25.5002 s]
/proc/loadavg: 4.54 4.25 4.14 5/267 90037
/proc/meminfo: memFree=121421784/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=86864 memory=77364 CPUtime=25.4 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 18909 0 0 0 2538 2 0 0 20 0 1 0 78684922 88948736 19341 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 21716 19341 1184 1162 0 20325 0
Current children cumulated CPU time: 25.4 s
Current children cumulated vsize: 91056 KiB
Current children cumulated memory: 80308 KiB

[startup+51.1054 s]
/proc/loadavg: 4.35 4.23 4.14 5/267 90038
/proc/meminfo: memFree=121395940/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=89936 memory=78972 CPUtime=50.93 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 20099 0 0 0 5090 3 0 0 20 0 1 0 78684922 92094464 19743 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 22484 19743 1184 1162 0 21093 0
Current children cumulated CPU time: 50.93 s
Current children cumulated vsize: 94128 KiB
Current children cumulated memory: 81916 KiB

[startup+102.3 s]
/proc/loadavg: 4.20 4.20 4.13 5/267 90040
/proc/meminfo: memFree=121371496/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=96080 memory=84436 CPUtime=101.96 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 21456 0 0 0 10193 3 0 0 20 0 1 0 78684922 98385920 21109 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 24020 21109 1184 1162 0 22629 0
Current children cumulated CPU time: 101.96 s
Current children cumulated vsize: 100272 KiB
Current children cumulated memory: 87380 KiB

[startup+162.3 s]
/proc/loadavg: 4.18 4.20 4.13 5/267 90042
/proc/meminfo: memFree=121320748/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=102224 memory=88588 CPUtime=161.77 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 21993 0 0 0 16174 3 0 0 20 0 1 0 78684922 104677376 22147 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 25556 22147 1184 1162 0 24165 0
Current children cumulated CPU time: 161.77 s
Current children cumulated vsize: 106416 KiB
Current children cumulated memory: 91532 KiB

[startup+222.3 s]
/proc/loadavg: 4.06 4.16 4.12 5/267 90049
/proc/meminfo: memFree=121327852/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=108368 memory=93228 CPUtime=221.57 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 22147 0 0 0 22154 3 0 0 20 0 1 0 78684922 110968832 23307 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 27092 23307 1184 1162 0 25701 0
Current children cumulated CPU time: 221.57 s
Current children cumulated vsize: 112560 KiB
Current children cumulated memory: 96172 KiB

################
# More data... #
################

/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 26410 0 0 0 339283 16 0 0 20 0 1 0 78684922 392241152 62503 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 95762 62503 1184 1162 0 94371 0
Current children cumulated CPU time: 3392.99 s
Current children cumulated vsize: 387240 KiB
Current children cumulated memory: 252956 KiB

[startup+3462.3 s]
/proc/loadavg: 2.19 3.04 3.58 2/246 90345
/proc/meminfo: memFree=121810284/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=383048 memory=252092 CPUtime=3452.85 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 26421 0 0 0 345269 16 0 0 20 0 1 0 78684922 392241152 63023 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 95762 63023 1184 1162 0 94371 0
Current children cumulated CPU time: 3452.85 s
Current children cumulated vsize: 387240 KiB
Current children cumulated memory: 255036 KiB

[startup+3522.3 s]
/proc/loadavg: 1.50 2.68 3.42 2/246 90347
/proc/meminfo: memFree=121809300/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=383048 memory=257140 CPUtime=3512.71 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 26430 0 0 0 351255 16 0 0 20 0 1 0 78684922 392241152 64285 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 95762 64285 1184 1162 0 94371 0
Current children cumulated CPU time: 3512.71 s
Current children cumulated vsize: 387240 KiB
Current children cumulated memory: 260084 KiB

[startup+3582.3 s]
/proc/loadavg: 1.23 2.39 3.27 2/246 90351
/proc/meminfo: memFree=121809716/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=383048 memory=259196 CPUtime=3572.57 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 26493 0 0 0 357241 16 0 0 20 0 1 0 78684922 392241152 64799 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 95762 64799 1184 1162 0 94371 0
Current children cumulated CPU time: 3572.57 s
Current children cumulated vsize: 387240 KiB
Current children cumulated memory: 262140 KiB

[startup+3609.47 s]
# the end of solver process 90035 was just reported to runsolver
# Child status: 0

# cumulated CPU time of all completed processes:  user=3599.53 s, system=0.172284 s

Solver just ended.
??? end of watcher thread
Dumping a history of the last processes samples

[startup+3584 s]*
/proc/loadavg: 1.21 2.36 3.25 2/246 90351
/proc/meminfo: memFree=121809716/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=383048 memory=259196 CPUtime=3574.27 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 26493 0 0 0 357411 16 0 0 20 0 1 0 78684922 392241152 64799 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 95762 64799 1184 1162 0 94371 0
Current children cumulated CPU time: 3574.27 s
Current children cumulated vsize: 387240 KiB
Current children cumulated memory: 262140 KiB

[startup+3596.8 s]
/proc/loadavg: 1.26 2.33 3.24 2/246 90351
/proc/meminfo: memFree=121809780/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=383048 memory=261268 CPUtime=3587.04 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 26495 0 0 0 358688 16 0 0 20 0 1 0 78684922 392241152 65317 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 95762 65317 1184 1162 0 94371 0
Current children cumulated CPU time: 3587.04 s
Current children cumulated vsize: 387240 KiB
Current children cumulated memory: 264212 KiB

[startup+3603.2 s]
/proc/loadavg: 1.24 2.31 3.22 2/246 90353
/proc/meminfo: memFree=121809812/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=383048 memory=261268 CPUtime=3593.42 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 26495 0 0 0 359326 16 0 0 20 0 1 0 78684922 392241152 65317 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 95762 65317 1184 1162 0 94371 0
Current children cumulated CPU time: 3593.42 s
Current children cumulated vsize: 387240 KiB
Current children cumulated memory: 264212 KiB

[startup+3606.4 s]
/proc/loadavg: 1.22 2.29 3.21 2/246 90353
/proc/meminfo: memFree=121809812/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=383048 memory=261268 CPUtime=3596.62 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 26496 0 0 0 359646 16 0 0 20 0 1 0 78684922 392241152 65317 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 95762 65317 1184 1162 0 94371 0
Current children cumulated CPU time: 3596.62 s
Current children cumulated vsize: 387240 KiB
Current children cumulated memory: 264212 KiB

[startup+3608 s]*
/proc/loadavg: 1.22 2.29 3.21 2/246 90353
/proc/meminfo: memFree=121809812/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=383048 memory=261268 CPUtime=3598.22 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 26496 0 0 0 359806 16 0 0 20 0 1 0 78684922 392241152 65317 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 95762 65317 1184 1162 0 94371 0
Current children cumulated CPU time: 3598.22 s
Current children cumulated vsize: 387240 KiB
Current children cumulated memory: 264212 KiB

[startup+3608.8 s]
/proc/loadavg: 1.22 2.29 3.21 2/246 90353
/proc/meminfo: memFree=121809812/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=383048 memory=261268 CPUtime=3599.01 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 26496 0 0 0 359885 16 0 0 20 0 1 0 78684922 392241152 65317 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 95762 65317 1184 1162 0 94371 0
Current children cumulated CPU time: 3599.01 s
Current children cumulated vsize: 387240 KiB
Current children cumulated memory: 264212 KiB

[startup+3609.2 s]
/proc/loadavg: 1.20 2.27 3.20 2/246 90353
/proc/meminfo: memFree=121809812/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=383048 memory=261268 CPUtime=3599.41 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 26496 0 0 0 359925 16 0 0 20 0 1 0 78684922 392241152 65317 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 95762 65317 1184 1162 0 94371 0
Current children cumulated CPU time: 3599.41 s
Current children cumulated vsize: 387240 KiB
Current children cumulated memory: 264212 KiB

[startup+3609.4 s]
/proc/loadavg: 1.20 2.27 3.20 2/246 90353
/proc/meminfo: memFree=121809812/131249996 swapFree=33010416/33010684
[pid=90035] ppid=90032 vsize=4192 memory=2944 CPUtime=0 cores=0,2
/proc/90035/stat : 90035 (toulbar2.sh) S 90032 90035 89411 0 -1 4194304 205 0 0 0 0 0 0 0 20 0 1 0 78684922 4292608 736 18446744073709551615 94010844614656 94010845509189 140724798103824 0 0 0 65536 4 65538 1 0 0 17 2 0 0 0 0 0 94010845748240 94010845795984 94010848571392 140724798105909 140724798106085 140724798106085 140724798111689 0
/proc/90035/statm: 1048 736 672 219 0 110 0
[pid=90037] ppid=90035 vsize=383048 memory=261268 CPUtime=3599.61 cores=0,2
/proc/90037/stat : 90037 (toulbar2) R 90035 90035 89411 0 -1 4194304 26496 0 0 0 359945 16 0 0 20 0 1 0 78684922 392241152 65317 18446744073709551615 4198400 8956741 140728415296432 0 0 0 0 0 33570818 0 0 0 17 0 0 0 0 0 0 9785056 9890480 18755584 140728415304051 140728415304170 140728415304170 140728415309772 0
/proc/90037/statm: 95762 65317 1184 1162 0 94371 0
Current children cumulated CPU time: 3599.61 s
Current children cumulated vsize: 387240 KiB
Current children cumulated memory: 264212 KiB

Child status: 0

Real time (s): 3609.47
CPU time (s): 3599.7
CPU user time (s): 3599.53
CPU system time (s): 0.172284
CPU usage (%): 99.7292
Max. virtual memory (cumulated for all children) (KiB): 387240
Max. memory (cumulated for all children) (KiB): 264212

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 3599.53
system time used= 0.172284
maximum resident set size= 331556
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 26712
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 87
involuntary context switches= 8006


# summary of solver processes directly reported to runsolver:
#   pid: 90035
#   total CPU time (s): 3599.7
#   total CPU user time (s): 3599.53
#   total CPU system time (s): 0.172284

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 6.73681 second user time and 14.6766 second system time

The end

Launcher Data 

Begin job on nodeC022 at 2025-07-04 13:54:32
IDJOB=4526354
IDBENCH=144045
IDSOLVER=3383
FILE ID=nodeC022/4526354-1751630072
RUNJOBID= nodeC022-1751626267-89421
SLURM_JOB_ID= 9598954
Free space on /tmp= 432368 MiB

SOLVER NAME= toulbar2 2025-06-13
BENCH NAME= PB24/WBO/PARTIAL-LIN/wcsp/spot5/normalized-spot5-412_wcsp.wbo
COMMAND LINE= toulbar2.sh toulbar2 BENCHNAME -timer=TIMELIMIT -hve=512 -f=3 -n: -precision=0
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4526354-1751630072/watcher-4526354-1751630072 -o /tmp/evaluation-result-4526354-1751630072/solver-4526354-1751630072 -C 3600 -W 3700 -M 31000 --bin-var /tmp/runsolver-exch-nodeC022-1751626267-89421 --watchdog 3760  toulbar2.sh toulbar2 HOME/instance-4526354-1751630072.wbo -timer=3600 -hve=512 -f=3 -n: -precision=0

TIME LIMIT= 3600 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 2

MD5SUM BENCH= 84bec45daf3e2f2cbd3688999a3e29ea
RANDOM SEED=1624738095

nodeC022 Linux 5.14.0-503.14.1.el9_5.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 15 12:04:32 UTC 2024

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3599.735
cache size	: 15360 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 pti ibrs ibpb stibp tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts vnmi
vmx flags	: vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple shadow_vmcs pml ept_violation_ve
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit mmio_stale_data
bogomips	: 6999.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3599.683
cache size	: 15360 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 pti ibrs ibpb stibp tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts vnmi
vmx flags	: vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple shadow_vmcs pml ept_violation_ve
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit mmio_stale_data
bogomips	: 6999.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3546.065
cache size	: 15360 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 pti ibrs ibpb stibp tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts vnmi
vmx flags	: vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple shadow_vmcs pml ept_violation_ve
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit mmio_stale_data
bogomips	: 6999.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3599.429
cache size	: 15360 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 pti ibrs ibpb stibp tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts vnmi
vmx flags	: vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple shadow_vmcs pml ept_violation_ve
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit mmio_stale_data
bogomips	: 6999.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3599.716
cache size	: 15360 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 pti ibrs ibpb stibp tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts vnmi
vmx flags	: vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple shadow_vmcs pml ept_violation_ve
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit mmio_stale_data
bogomips	: 6999.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3599.709
cache size	: 15360 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 pti ibrs ibpb stibp tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts vnmi
vmx flags	: vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple shadow_vmcs pml ept_violation_ve
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit mmio_stale_data
bogomips	: 6999.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3700.000
cache size	: 15360 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 pti ibrs ibpb stibp tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts vnmi
vmx flags	: vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple shadow_vmcs pml ept_violation_ve
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit mmio_stale_data
bogomips	: 6999.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3599.887
cache size	: 15360 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 pti ibrs ibpb stibp tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts vnmi
vmx flags	: vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple shadow_vmcs pml ept_violation_ve
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit mmio_stale_data
bogomips	: 6999.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       131249996 kB
MemFree:        121512176 kB
MemAvailable:   129118596 kB
Buffers:            2128 kB
Cached:          8357288 kB
SwapCached:           32 kB
Active:          1329428 kB
Inactive:        7706312 kB
Active(anon):     690180 kB
Inactive(anon):     4176 kB
Active(file):     639248 kB
Inactive(file):  7702136 kB
Unevictable:        3072 kB
Mlocked:               0 kB
SwapTotal:      33010684 kB
SwapFree:       33010416 kB
Zswap:                 0 kB
Zswapped:              0 kB
Dirty:              6488 kB
Writeback:             0 kB
AnonPages:        679372 kB
Mapped:           118224 kB
Shmem:             18028 kB
KReclaimable:     208152 kB
Slab:             299816 kB
SReclaimable:     208152 kB
SUnreclaim:        91664 kB
KernelStack:        4304 kB
PageTables:         4468 kB
SecPageTables:         0 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    98635680 kB
Committed_AS:    1068848 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      150568 kB
VmallocChunk:          0 kB
Percpu:             3776 kB
HardwareCorrupted:     0 kB
AnonHugePages:    223232 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
FileHugePages:         0 kB
FilePmdMapped:         0 kB
CmaTotal:              0 kB
CmaFree:               0 kB
Unaccepted:            0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB
DirectMap4k:      205524 kB
DirectMap2M:     7954432 kB
DirectMap1G:    127926272 kB

Free space on /tmp at the end= 432384 MiB
End job on nodeC022 at 2025-07-04 14:54:41