Trace number 4103889

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
toysat 2016-05-02? (exit code) 870.51 870.657

General information on the benchmark

Namenormalized-PB06/OPT-BIGINT/mps-v2-20-10/MIPLIB/
miplib2003/normalized-mps-v2-20-10-momentum2.opb
MD5SUM0d80b4a073c50ac034a87a4950dc06d8
Bench CategoryOPT-BIGINT (optimisation, big integers)
Best result obtained on this benchmarkUNSAT
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark1.17882
Has Objective FunctionYES
SatisfiableNO
(Un)Satisfiability was provedYES
Best value of the objective function
Optimality of the best value was proved
Number of variables25930
Total number of constraints22668
Number of constraints which are clauses10362
Number of constraints which are cardinality constraints (but not clauses)103
Number of constraints which are nor clauses,nor cardinality constraints12203
Minimum length of a constraint1
Maximum length of a constraint913
Number of terms in the objective function 24130
Biggest coefficient in the objective function 655360000
Number of bits for the biggest coefficient in the objective function 30
Sum of the numbers in the objective function 3638660312
Number of bits of the sum of numbers in the objective function 32
Biggest number in a constraint 1024000000000000000000000000000
Number of bits of the biggest number in a constraint 100
Biggest sum of numbers in a constraint 3743132712137090008913823010064
Number of bits of the biggest sum of numbers102
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c 2016-06-18T10:43:57+0200
0.00/0.00	c version = 0.5.0
0.00/0.00	c githash = 1a49202
0.00/0.00	c compilationtime = 2016-05-02 00:19:28.990761 UTC
0.00/0.00	c arch = x86_64
0.00/0.00	c os = linux
0.00/0.00	c compiler = ghc 7.10
0.00/0.00	c packages:
0.00/0.00	c   OptDir-0.0.3
0.00/0.00	c   array-0.5.1.0
0.00/0.00	c   base-4.8.2.0
0.00/0.00	c   bytestring-0.10.6.0
0.00/0.00	c   bytestring-builder-0.10.6.0.0
0.00/0.00	c   clock-0.7.2
0.00/0.00	c   containers-0.5.6.2
0.00/0.00	c   data-default-0.5.3
0.00/0.00	c   data-default-class-0.0.1
0.00/0.00	c   data-interval-1.2.0
0.00/0.00	c   deepseq-1.4.1.1
0.00/0.00	c   directory-1.2.2.0
0.00/0.00	c   exceptions-0.8.2.1
0.00/0.00	c   extended-reals-0.2.1.0
0.00/0.00	c   filepath-1.4.0.0
0.00/0.00	c   finite-field-0.8.0
0.00/0.00	c   hashable-1.2.4.0
0.00/0.00	c   heaps-0.3.3
0.00/0.00	c   intern-0.9.1.4
0.00/0.00	c   loop-0.3.0
0.00/0.00	c   mtl-2.2.1
0.00/0.00	c   multiset-0.3.2
0.00/0.00	c   mwc-random-0.13.3.2
0.00/0.00	c   parse_dimacs-1.3
0.00/0.00	c   parsec-3.1.9
0.00/0.00	c   prettyclass-1.0.0.0
0.00/0.00	c   primes-0.2.1.0
0.00/0.00	c   primitive-0.6.1.0
0.00/0.00	c   process-1.2.3.0
0.00/0.00	c   pseudo-boolean-0.1.3.0
0.00/0.01	c   queue-0.1.2
0.00/0.01	c   semigroups-0.18.1
0.00/0.01	c   sign-0.4.3
0.00/0.01	c   stm-2.4.4.1
0.00/0.01	c   template-haskell-2.10.0.0
0.00/0.01	c   temporary-1.2.0.4
0.00/0.01	c   time-1.5.0.1
0.00/0.01	c   transformers-0.4.2.0
0.00/0.01	c   transformers-compat-0.5.1.4
0.00/0.01	c   type-level-numbers-0.1.1.1
0.00/0.01	c   unbounded-delays-0.1.0.9
0.00/0.01	c   unordered-containers-0.2.6.0
0.00/0.01	c   vector-0.11.0.0
0.00/0.01	c   vector-space-0.10.3
0.00/0.01	c command line = ["+RTS","-H1G","-M15500m","-K1G","-RTS","--pb","--search=bcd2","HOME/instance-4103889-1466239435.opb"]
0.00/0.01	c use --random-gen="1221572123 1204691598 899432402 2574857721 535229895 3049838906 1646512481 1898006048 1465738154 3778990562 1786715710 1680757173 2342872489 1011168554 3256755969 877169813 2646379692 1822532828 1671560181 426787183 2967829824 2581124606 2902185813 3318039380 1788548982 33289053 3753812483 811516613 2541240840 3711982936 62255964 3210638572 211167663 2871898258 3899602002 4171150463 307148635 3228554339 3255756944 2110334951 1721540495 3948574546 3753727099 2525018129 2990406457 527660854 644722736 1165111952 3167492365 1130034463 161803566 4077142413 3367732775 2290207262 2833423280 2358093240 4125089205 3483933435 1880388193 458383781 1678170592 3977093567 1429515747 791039398 1964801139 2086890748 310762706 2701567084 1688982827 2974408415 4195224958 425274122 563720194 718795572 1261526530 975877235 1080228255 2687112186 949490150 1824373093 244448732 2814011188 1524149471 2441877231 3963287088 3952066219 96055884 333067015 1387985101 2465013191 2419563215 545520372 3040066091 4127103489 981425689 3603389369 2071730774 1858880865 2498437256 1115487527 2039964362 23583839 1981446834 3781392281 2991227987 1473985079 396917946 2916558475 2189069716 1189912775 1170934491 4059238984 1223473973 764579005 1814229959 2842098346 3066576820 3147900314 2874019371 825832518 4002272814 1003737142 1801635351 3017248511 3354697390 3900028571 2531112355 3017933945 4223111636 3624067725 852093686 361523294 449714180 1777549847 3121063037 667801428 1130840535 581035153 919178191 1305240114 443110736 3629221621 4095240855 770204527 1116114976 3891194994 1307525657 698009054 1352844838 270306155 1938998698 3135421850 432707034 2308565678 2907717916 4138166422 1903946249 263456906 2057019261 3334395437 4065609048 3852222755 3990176132 2847204412 1621586914 3480965510 2432450481 1992351462 3727896826 1657800918 1488332579 2055414517 1974506897 1719477982 207093437 1814183180 2376523358 4025225280 3249705475 2242694038 3022423703 2226934230 257120862 2611649571 3532910742 401237763 941074762 786551606 2429362367 407274423 3296021999 4194631156 3729257250 3302518575 1663547769 331013470 2088920169 3396238209 501390317 513181833 2972476125 3543534604 3214768988 479868813 792520498 4199914149 373742426 1871698552 38339831 744764372 3498934248 64779569 3878160722 3169992239 173185485 2271438520 1371160007 2365494242 1043623332 1932326521 17028528 1675528765 442069015 2201430706 911053785 3048943335 2420633018 196259244 2484273556 1857728409 158553842 2544743938 1177702782 1742947926 3058938652 1516159170 3406981014 2493321417 230966846 800605842 2822164669 214802739 1412457063 239382917 1101765879 115553953 1277160576 887822926 2087414080 4057951853 4051616849 1157158762 1792022675 431775282 836301089 1085865264 255 362436" option to reproduce the execution
6.99/7.01	c #vars 25930
6.99/7.01	c #constraints 22668
869.64/869.87	toysat: out of memory (requested 244318208 bytes)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2013 Olivier ROUSSEL

This is runsolver version 3.3.7 (svn: 2553)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4103889-1466239435/watcher-4103889-1466239435 -o /tmp/evaluation-result-4103889-1466239435/solver-4103889-1466239435 -C 1800 -W 1900 -M 15500 --bin-var /tmp/runsolver-exch-node123-1466206081-595 --watchdog 1960 toysat +RTS -H1G -M15500m -K1G -RTS --pb --search=bcd2 HOME/instance-4103889-1466239435.opb 

pid=6363
running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.19 2.26 2.25 3/179 6366
/proc/meminfo: memFree=24372368/32872528 swapFree=17888/1798212
[pid=6366] ppid=6363 vsize=731580 CPUtime=0 cores=0,2,4,6
/proc/6366/stat : 6366 (toysat) R 6363 6366 579 0 -1 4202496 2730 0 0 0 0 0 0 0 20 0 4 0 25990846 749137920 3091 33554432000 4194304 16246674 140735977887984 140735977887408 13880264 0 2 0 34082818 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6366/statm: 183407 3101 441 2943 0 180301 0
[pid=6366/tid=6367] ppid=6363 vsize=750012 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6367/stat : 6367 (toysat) S 6363 6366 579 0 -1 4202560 10 0 0 0 0 0 0 0 20 0 4 0 25990846 768012288 3163 33554432000 4194304 16246674 140735977887984 140625845398768 15012391 0 0 0 34082818 18446744071580610281 0 0 -1 4 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=754108 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) S 6363 6366 579 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 4 0 25990846 772206592 3180 33554432000 4194304 16246674 140735977887984 140625834908912 14998865 0 0 0 34082818 18446744071580425449 0 0 -1 0 0 0 0 0 0
[pid=6366/tid=6369] ppid=6363 vsize=758204 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6369/stat : 6369 (toysat) S 6363 6366 579 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 4 0 25990846 776400896 3196 33554432000 4194304 16246674 140735977887984 140625752091824 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 4 0 0 0 0 0

[startup+0.100169 s]
/proc/loadavg: 2.19 2.26 2.25 3/179 6366
/proc/meminfo: memFree=24372368/32872528 swapFree=17888/1798212
[pid=6366] ppid=6363 vsize=1303108 CPUtime=0.09 cores=0,2,4,6
/proc/6366/stat : 6366 (toysat) R 6363 6366 579 0 -1 4202496 29134 0 0 0 6 3 0 0 20 0 4 0 25990846 1334382592 29433 33554432000 4194304 16246674 140735977887984 140735977887296 7364033 0 0 0 34082818 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6366/statm: 325777 29447 798 2943 0 322671 0
[pid=6366/tid=6367] ppid=6363 vsize=1303108 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6367/stat : 6367 (toysat) S 6363 6366 579 0 -1 4202560 10 0 0 0 0 0 0 0 20 0 4 0 25990846 1334382592 29458 33554432000 4194304 16246674 140735977887984 140625845398768 15012391 0 0 0 34082818 18446744071580610281 0 0 -1 4 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=1303108 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) S 6363 6366 579 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 4 0 25990846 1334382592 29469 33554432000 4194304 16246674 140735977887984 140625834908912 14998865 0 0 0 34082818 18446744071580425449 0 0 -1 0 0 0 0 0 0
[pid=6366/tid=6369] ppid=6363 vsize=1303108 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6369/stat : 6369 (toysat) S 6363 6366 579 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 4 0 25990846 1334382592 29478 33554432000 4194304 16246674 140735977887984 140625752091824 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 1303108

[startup+0.200473 s]
/proc/loadavg: 2.19 2.26 2.25 3/179 6366
/proc/meminfo: memFree=24372368/32872528 swapFree=17888/1798212
[pid=6366] ppid=6363 vsize=1311300 CPUtime=0.19 cores=0,2,4,6
/proc/6366/stat : 6366 (toysat) R 6363 6366 579 0 -1 4202496 40686 0 0 0 14 5 0 0 20 0 4 0 25990846 1342771200 41495 33554432000 4194304 16246674 140735977887984 140735977887296 7374601 0 0 0 34082818 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6366/statm: 327825 41495 800 2943 0 324719 0
[pid=6366/tid=6367] ppid=6363 vsize=1311300 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6367/stat : 6367 (toysat) S 6363 6366 579 0 -1 4202560 10 0 0 0 0 0 0 0 20 0 4 0 25990846 1342771200 41495 33554432000 4194304 16246674 140735977887984 140625845398768 15012391 0 0 0 34082818 18446744071580610281 0 0 -1 4 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=1311300 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) S 6363 6366 579 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 4 0 25990846 1342771200 41495 33554432000 4194304 16246674 140735977887984 140625834908912 14998865 0 0 0 34082818 18446744071580425449 0 0 -1 0 0 0 0 0 0
[pid=6366/tid=6369] ppid=6363 vsize=1311300 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6369/stat : 6369 (toysat) S 6363 6366 579 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 4 0 25990846 1342771200 41495 33554432000 4194304 16246674 140735977887984 140625752091824 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 1311300

[startup+0.300517 s]
/proc/loadavg: 2.19 2.26 2.25 3/179 6366
/proc/meminfo: memFree=24372368/32872528 swapFree=17888/1798212
[pid=6366] ppid=6363 vsize=1311300 CPUtime=0.29 cores=0,2,4,6
/proc/6366/stat : 6366 (toysat) R 6363 6366 579 0 -1 4202496 40687 0 0 0 24 5 0 0 20 0 4 0 25990846 1342771200 41496 33554432000 4194304 16246674 140735977887984 140735977870976 4220897 0 2 0 34082818 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6366/statm: 327825 41496 801 2943 0 324719 0
[pid=6366/tid=6367] ppid=6363 vsize=1311300 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6367/stat : 6367 (toysat) S 6363 6366 579 0 -1 4202560 10 0 0 0 0 0 0 0 20 0 4 0 25990846 1342771200 41496 33554432000 4194304 16246674 140735977887984 140625845398768 15012391 0 0 0 34082818 18446744071580610281 0 0 -1 4 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=1311300 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) S 6363 6366 579 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 4 0 25990846 1342771200 41496 33554432000 4194304 16246674 140735977887984 140625834908912 14998865 0 0 0 34082818 18446744071580425449 0 0 -1 0 0 0 0 0 0
[pid=6366/tid=6369] ppid=6363 vsize=1311300 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6369/stat : 6369 (toysat) S 6363 6366 579 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 4 0 25990846 1342771200 41496 33554432000 4194304 16246674 140735977887984 140625752091824 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 1311300

[startup+0.700441 s]
/proc/loadavg: 2.19 2.26 2.25 3/179 6366
/proc/meminfo: memFree=24372368/32872528 swapFree=17888/1798212
[pid=6366] ppid=6363 vsize=1312324 CPUtime=0.69 cores=0,2,4,6
/proc/6366/stat : 6366 (toysat) R 6363 6366 579 0 -1 4202496 56653 0 0 0 62 7 0 0 20 0 4 0 25990846 1343819776 57462 33554432000 4194304 16246674 140735977887984 140735977870976 4206649 0 2 0 34082818 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6366/statm: 328081 57462 802 2943 0 324975 0
[pid=6366/tid=6367] ppid=6363 vsize=1312324 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6367/stat : 6367 (toysat) S 6363 6366 579 0 -1 4202560 10 0 0 0 0 0 0 0 20 0 4 0 25990846 1343819776 57462 33554432000 4194304 16246674 140735977887984 140625845398768 15012391 0 0 0 34082818 18446744071580610281 0 0 -1 4 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=1312324 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) S 6363 6366 579 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 4 0 25990846 1343819776 57462 33554432000 4194304 16246674 140735977887984 140625834908912 14998865 0 0 0 34082818 18446744071580425449 0 0 -1 0 0 0 0 0 0
[pid=6366/tid=6369] ppid=6363 vsize=1312324 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6369/stat : 6369 (toysat) S 6363 6366 579 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 4 0 25990846 1343819776 57462 33554432000 4194304 16246674 140735977887984 140625752091824 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 1312324

[startup+1.50041 s]
/proc/loadavg: 2.19 2.26 2.25 3/183 6370
/proc/meminfo: memFree=23174968/32872528 swapFree=17888/1798212
[pid=6366] ppid=6363 vsize=1349188 CPUtime=1.48 cores=0,2,4,6
/proc/6366/stat : 6366 (toysat) R 6363 6366 579 0 -1 4202496 72608 0 0 0 139 9 0 0 20 0 4 0 25990846 1381568512 80060 33554432000 4194304 16246674 140735977887984 140735977885888 4220880 0 2 0 34082818 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6366/statm: 337297 80060 802 2943 0 334191 0
[pid=6366/tid=6367] ppid=6363 vsize=1349188 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6367/stat : 6367 (toysat) S 6363 6366 579 0 -1 4202560 10 0 0 0 0 0 0 0 20 0 4 0 25990846 1381568512 80060 33554432000 4194304 16246674 140735977887984 140625845398768 15012391 0 0 0 34082818 18446744071580610281 0 0 -1 4 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=1349188 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) S 6363 6366 579 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 4 0 25990846 1381568512 80060 33554432000 4194304 16246674 140735977887984 140625834908912 14998865 0 0 0 34082818 18446744071580425449 0 0 -1 0 0 0 0 0 0
[pid=6366/tid=6369] ppid=6363 vsize=1349188 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6369/stat : 6369 (toysat) S 6363 6366 579 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 4 0 25990846 1381568512 80060 33554432000 4194304 16246674 140735977887984 140625752091824 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 1349188

[startup+3.10042 s]
/proc/loadavg: 2.19 2.26 2.25 3/183 6370
/proc/meminfo: memFree=22860248/32872528 swapFree=17888/1798212
[pid=6366] ppid=6363 vsize=1398340 CPUtime=3.09 cores=0,2,4,6
/proc/6366/stat : 6366 (toysat) R 6363 6366 579 0 -1 4202496 96097 0 0 0 295 14 0 0 20 0 4 0 25990846 1431900160 113769 33554432000 4194304 16246674 140735977887984 140735977885824 4220880 0 2 0 34082818 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6366/statm: 349585 113769 802 2943 0 346479 0
[pid=6366/tid=6367] ppid=6363 vsize=1398340 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6367/stat : 6367 (toysat) S 6363 6366 579 0 -1 4202560 10 0 0 0 0 0 0 0 20 0 4 0 25990846 1431900160 113769 33554432000 4194304 16246674 140735977887984 140625845398768 15012391 0 0 0 34082818 18446744071580610281 0 0 -1 4 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=1398340 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) S 6363 6366 579 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 4 0 25990846 1431900160 113769 33554432000 4194304 16246674 140735977887984 140625834908912 14998865 0 0 0 34082818 18446744071580425449 0 0 -1 0 0 0 0 0 0
[pid=6366/tid=6369] ppid=6363 vsize=1398340 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6369/stat : 6369 (toysat) S 6363 6366 579 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 4 0 25990846 1431900160 113769 33554432000 4194304 16246674 140735977887984 140625752091824 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 1398340

[startup+6.3004 s]
/proc/loadavg: 2.17 2.25 2.25 3/183 6370
/proc/meminfo: memFree=22583164/32872528 swapFree=17888/1798212
[pid=6366] ppid=6363 vsize=1496644 CPUtime=6.29 cores=0,2,4,6
/proc/6366/stat : 6366 (toysat) R 6363 6366 579 0 -1 4202496 184222 0 0 0 602 27 0 0 20 0 4 0 25990846 1532563456 221312 33554432000 4194304 16246674 140735977887984 140735977885824 4221105 0 2 0 34082818 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6366/statm: 374161 221319 802 2943 0 371055 0
[pid=6366/tid=6367] ppid=6363 vsize=1496644 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6367/stat : 6367 (toysat) S 6363 6366 579 0 -1 4202560 10 0 0 0 0 0 0 0 20 0 4 0 25990846 1532563456 221323 33554432000 4194304 16246674 140735977887984 140625845398768 15012391 0 0 0 34082818 18446744071580610281 0 0 -1 4 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=1496644 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) S 6363 6366 579 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 4 0 25990846 1532563456 221328 33554432000 4194304 16246674 140735977887984 140625834908688 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=6366/tid=6369] ppid=6363 vsize=1496644 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6369/stat : 6369 (toysat) S 6363 6366 579 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 4 0 25990846 1532563456 221332 33554432000 4194304 16246674 140735977887984 140625752091824 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 1496644

[startup+12.7004 s]
/proc/loadavg: 2.24 2.26 2.26 4/183 6370
/proc/meminfo: memFree=20029508/32872528 swapFree=17888/1798212
[pid=6366] ppid=6363 vsize=1496644 CPUtime=12.69 cores=0,2,4,6
/proc/6366/stat : 6366 (toysat) R 6363 6366 579 0 -1 4202496 241561 0 0 0 1234 35 0 0 20 0 4 0 25990846 1532563456 280184 33554432000 4194304 16246674 140735977887984 140735977869888 4220897 0 2 0 34082818 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6366/statm: 374161 280184 843 2943 0 371055 0
[pid=6366/tid=6367] ppid=6363 vsize=1496644 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6367/stat : 6367 (toysat) S 6363 6366 579 0 -1 4202560 10 0 0 0 0 0 0 0 20 0 4 0 25990846 1532563456 280184 33554432000 4194304 16246674 140735977887984 140625845398768 15012391 0 0 0 34082818 18446744071580610281 0 0 -1 4 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=1496644 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) S 6363 6366 579 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 4 0 25990846 1532563456 280184 33554432000 4194304 16246674 140735977887984 140625834908912 14998865 0 0 0 34082818 18446744071580425449 0 0 -1 0 0 0 0 0 0
[pid=6366/tid=6369] ppid=6363 vsize=1496644 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6369/stat : 6369 (toysat) S 6363 6366 579 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 4 0 25990846 1532563456 280184 33554432000 4194304 16246674 140735977887984 140625752091824 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 1496644

[startup+25.5074 s]
/proc/loadavg: 2.33 2.28 2.26 3/183 6370
/proc/meminfo: memFree=25088788/32872528 swapFree=17888/1798212
[pid=6366] ppid=6363 vsize=2289220 CPUtime=25.49 cores=0,2,4,6
/proc/6366/stat : 6366 (toysat) R 6363 6366 579 0 -1 4202496 466941 0 0 0 2484 65 0 0 20 0 4 0 25990846 2344161280 513740 33554432000 4194304 16246674 140735977887984 140735977885888 4223652 0 2 0 34082818 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6366/statm: 572305 513740 844 2943 0 569199 0
[pid=6366/tid=6367] ppid=6363 vsize=2289220 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6367/stat : 6367 (toysat) S 6363 6366 579 0 -1 4202560 10 0 0 0 0 0 0 0 20 0 4 0 25990846 2344161280 513740 33554432000 4194304 16246674 140735977887984 140625845398768 15012391 0 0 0 34082818 18446744071580610281 0 0 -1 4 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=2289220 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) S 6363 6366 579 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 4 0 25990846 2344161280 513740 33554432000 4194304 16246674 140735977887984 140625834908912 14998865 0 0 0 34082818 18446744071580425449 0 0 -1 0 0 0 0 0 0
[pid=6366/tid=6369] ppid=6363 vsize=2289220 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6369/stat : 6369 (toysat) S 6363 6366 579 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 4 0 25990846 2344161280 513740 33554432000 4194304 16246674 140735977887984 140625752091824 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 25.49
Current children cumulated vsize (KiB) 2289220

[startup+51.1005 s]
/proc/loadavg: 2.35 2.29 2.27 3/183 6370
/proc/meminfo: memFree=24919624/32872528 swapFree=17888/1798212
[pid=6366] ppid=6363 vsize=2458180 CPUtime=51.08 cores=0,2,4,6
/proc/6366/stat : 6366 (toysat) R 6363 6366 579 0 -1 4202496 509181 0 0 0 5035 73 0 0 20 0 4 0 25990846 2517176320 555980 33554432000 4194304 16246674 140735977887984 140735977869888 12654139 0 0 0 34082818 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6366/statm: 614545 555980 844 2943 0 611439 0
[pid=6366/tid=6367] ppid=6363 vsize=2458180 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6367/stat : 6367 (toysat) S 6363 6366 579 0 -1 4202560 10 0 0 0 0 0 0 0 20 0 4 0 25990846 2517176320 555980 33554432000 4194304 16246674 140735977887984 140625845398768 15012391 0 0 0 34082818 18446744071580610281 0 0 -1 4 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=2458180 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) S 6363 6366 579 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 4 0 25990846 2517176320 555980 33554432000 4194304 16246674 140735977887984 140625834908912 14998865 0 0 0 34082818 18446744071580425449 0 0 -1 0 0 0 0 0 0
[pid=6366/tid=6369] ppid=6363 vsize=2458180 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6369/stat : 6369 (toysat) S 6363 6366 579 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 4 0 25990846 2517176320 555980 33554432000 4194304 16246674 140735977887984 140625752091824 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 51.08
Current children cumulated vsize (KiB) 2458180

[startup+102.3 s]
/proc/loadavg: 2.31 2.29 2.27 3/183 6371
/proc/meminfo: memFree=19214684/32872528 swapFree=17888/1798212
[pid=6366] ppid=6363 vsize=2458180 CPUtime=102.27 cores=0,2,4,6
/proc/6366/stat : 6366 (toysat) R 6363 6366 579 0 -1 4202496 509181 0 0 0 10151 76 0 0 20 0 4 0 25990846 2517176320 555980 33554432000 4194304 16246674 140735977887984 140735977869888 12371230 0 0 0 34082818 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6366/statm: 614545 555980 844 2943 0 611439 0
[pid=6366/tid=6367] ppid=6363 vsize=2458180 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6367/stat : 6367 (toysat) S 6363 6366 579 0 -1 4202560 10 0 0 0 0 0 0 0 20 0 4 0 25990846 2517176320 555980 33554432000 4194304 16246674 140735977887984 140625845398768 15012391 0 0 0 34082818 18446744071580610281 0 0 -1 4 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=2458180 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) S 6363 6366 579 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 4 0 25990846 2517176320 555980 33554432000 4194304 16246674 140735977887984 140625834908912 14998865 0 0 0 34082818 18446744071580425449 0 0 -1 0 0 0 0 0 0
[pid=6366/tid=6369] ppid=6363 vsize=2458180 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6369/stat : 6369 (toysat) S 6363 6366 579 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 4 0 25990846 2517176320 555980 33554432000 4194304 16246674 140735977887984 140625752091824 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 102.27
Current children cumulated vsize (KiB) 2458180


################
# More data... #
################

/proc/6366/task/6367/stat : 6367 (toysat) S 6363 6366 579 0 -1 4202560 10 0 0 0 0 0 0 0 20 0 4 0 25990846 2796097536 624343 33554432000 4194304 16246674 140735977887984 140625845398768 15012391 0 0 0 34082818 18446744071580610281 0 0 -1 4 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=2730564 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) S 6363 6366 579 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 4 0 25990846 2796097536 624343 33554432000 4194304 16246674 140735977887984 140625834908912 14998865 0 0 0 34082818 18446744071580425449 0 0 -1 0 0 0 0 0 0
[pid=6366/tid=6369] ppid=6363 vsize=2730564 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6369/stat : 6369 (toysat) S 6363 6366 579 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 4 0 25990846 2796097536 624343 33554432000 4194304 16246674 140735977887984 140625752091824 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 702.17
Current children cumulated vsize (KiB) 2730564

[startup+762.3 s]
/proc/loadavg: 1.28 1.67 2.00 2/173 6466
/proc/meminfo: memFree=20581540/32872528 swapFree=17888/1798212
[pid=6366] ppid=6363 vsize=9774660 CPUtime=762.15 cores=0,2,4,6
/proc/6366/stat : 6366 (toysat) R 6363 6366 579 0 -1 4202496 2425350 0 0 0 75847 368 0 0 20 0 4 0 25990846 10009251840 2385371 33554432000 4194304 16246674 140735977887984 140735977885824 4220489 0 2 0 34082818 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6366/statm: 2443665 2385371 850 2943 0 2440559 0
[pid=6366/tid=6367] ppid=6363 vsize=9774660 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6367/stat : 6367 (toysat) S 6363 6366 579 0 -1 4202560 10 0 0 0 0 0 0 0 20 0 4 0 25990846 10009251840 2385371 33554432000 4194304 16246674 140735977887984 140625845398768 15012391 0 0 0 34082818 18446744071580610281 0 0 -1 4 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=9774660 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) S 6363 6366 579 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 4 0 25990846 10009251840 2385371 33554432000 4194304 16246674 140735977887984 140625834908688 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=6366/tid=6369] ppid=6363 vsize=9774660 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6369/stat : 6369 (toysat) S 6363 6366 579 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 4 0 25990846 10009251840 2385371 33554432000 4194304 16246674 140735977887984 140625752091824 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 762.15
Current children cumulated vsize (KiB) 9774660

[startup+822.3 s]
/proc/loadavg: 1.32 1.60 1.96 2/173 6467
/proc/meminfo: memFree=17324908/32872528 swapFree=17888/1798212
[pid=6366] ppid=6363 vsize=13026884 CPUtime=822.15 cores=0,2,4,6
/proc/6366/stat : 6366 (toysat) R 6363 6366 579 0 -1 4202496 3190046 0 0 0 81735 480 0 0 20 0 4 0 25990846 13339529216 3198101 33554432000 4194304 16246674 140735977887984 140735977885952 4220489 0 2 0 34082818 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6366/statm: 3256721 3198101 855 2943 0 3253615 0
[pid=6366/tid=6367] ppid=6363 vsize=13026884 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6367/stat : 6367 (toysat) S 6363 6366 579 0 -1 4202560 10 0 0 0 0 0 0 0 20 0 4 0 25990846 13339529216 3198101 33554432000 4194304 16246674 140735977887984 140625845398768 15012391 0 0 0 34082818 18446744071580610281 0 0 -1 4 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=13026884 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) S 6363 6366 579 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 4 0 25990846 13339529216 3198101 33554432000 4194304 16246674 140735977887984 140625834908688 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=6366/tid=6369] ppid=6363 vsize=13026884 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6369/stat : 6369 (toysat) S 6363 6366 579 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 4 0 25990846 13339529216 3198101 33554432000 4194304 16246674 140735977887984 140625752091824 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 822.15
Current children cumulated vsize (KiB) 13026884

Solver just ended.
??? exit watcher main loop
??? sigkill at end of watcher main loop
??? kill 6366 sig 9
??? end of watcher thread
Dumping a history of the last processes samples

[startup+844.9 s]
/proc/loadavg: 1.33 1.59 1.94 2/173 6467
/proc/meminfo: memFree=17324516/32872528 swapFree=17888/1798212
[pid=6366] ppid=6363 vsize=13026884 CPUtime=844.75 cores=0,2,4,6
/proc/6366/stat : 6366 (toysat) R 6363 6366 579 0 -1 4202496 3190073 0 0 0 83995 480 0 0 20 0 4 0 25990846 13339529216 3198128 33554432000 4194304 16246674 140735977887984 140735977885952 14014086 0 2 0 34082818 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6366/statm: 3256721 3198128 855 2943 0 3253615 0
[pid=6366/tid=6367] ppid=6363 vsize=13026884 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6367/stat : 6367 (toysat) S 6363 6366 579 0 -1 4202560 10 0 0 0 0 0 0 0 20 0 4 0 25990846 13339529216 3198128 33554432000 4194304 16246674 140735977887984 140625845398768 15012391 0 0 0 34082818 18446744071580610281 0 0 -1 4 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=13026884 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) S 6363 6366 579 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 4 0 25990846 13339529216 3198128 33554432000 4194304 16246674 140735977887984 140625834908688 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=6366/tid=6369] ppid=6363 vsize=13026884 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6369/stat : 6369 (toysat) S 6363 6366 579 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 4 0 25990846 13339529216 3198128 33554432000 4194304 16246674 140735977887984 140625752091824 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 844.75
Current children cumulated vsize (KiB) 13026884

[startup+857.7 s]
/proc/loadavg: 1.35 1.58 1.94 3/173 6468
/proc/meminfo: memFree=17323072/32872528 swapFree=17888/1798212
[pid=6366] ppid=6363 vsize=13026884 CPUtime=857.54 cores=0,2,4,6
/proc/6366/stat : 6366 (toysat) R 6363 6366 579 0 -1 4202496 3190378 0 0 0 85273 481 0 0 20 0 4 0 25990846 13339529216 3198433 33554432000 4194304 16246674 140735977887984 140735977887160 4220543 0 2 0 34082818 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6366/statm: 3256721 3198433 855 2943 0 3253615 0
[pid=6366/tid=6367] ppid=6363 vsize=13026884 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6367/stat : 6367 (toysat) S 6363 6366 579 0 -1 4202560 10 0 0 0 0 0 0 0 20 0 4 0 25990846 13339529216 3198433 33554432000 4194304 16246674 140735977887984 140625845398768 15012391 0 0 0 34082818 18446744071580610281 0 0 -1 4 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=13026884 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) S 6363 6366 579 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 4 0 25990846 13339529216 3198433 33554432000 4194304 16246674 140735977887984 140625834908912 14998865 0 0 0 34082818 18446744071580425449 0 0 -1 0 0 0 0 0 0
[pid=6366/tid=6369] ppid=6363 vsize=13026884 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6369/stat : 6369 (toysat) S 6363 6366 579 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 4 0 25990846 13339529216 3198433 33554432000 4194304 16246674 140735977887984 140625752091824 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 857.54
Current children cumulated vsize (KiB) 13026884

[startup+864.1 s]
/proc/loadavg: 1.33 1.57 1.93 2/173 6468
/proc/meminfo: memFree=17237372/32872528 swapFree=17888/1798212
[pid=6366] ppid=6363 vsize=13320772 CPUtime=863.94 cores=0,2,4,6
/proc/6366/stat : 6366 (toysat) R 6363 6366 579 0 -1 4202496 3263724 0 0 0 85899 495 0 0 20 0 4 0 25990846 13640470528 3271779 33554432000 4194304 16246674 140735977887984 140735977887160 4225154 0 2 0 34082818 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6366/statm: 3330193 3271789 855 2943 0 3327087 0
[pid=6366/tid=6367] ppid=6363 vsize=13320772 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6367/stat : 6367 (toysat) S 6363 6366 579 0 -1 4202560 10 0 0 0 0 0 0 0 20 0 4 0 25990846 13640470528 3271795 33554432000 4194304 16246674 140735977887984 140625845398768 15012391 0 0 0 34082818 18446744071580610281 0 0 -1 4 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=13320772 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) S 6363 6366 579 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 4 0 25990846 13640470528 3271800 33554432000 4194304 16246674 140735977887984 140625834908912 14998865 0 0 0 34082818 18446744071580425449 0 0 -1 0 0 0 0 0 0
[pid=6366/tid=6369] ppid=6363 vsize=13320772 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6369/stat : 6369 (toysat) S 6363 6366 579 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 4 0 25990846 13640470528 3271805 33554432000 4194304 16246674 140735977887984 140625752091824 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 863.94
Current children cumulated vsize (KiB) 13320772

[startup+867.3 s]
/proc/loadavg: 1.30 1.56 1.93 2/173 6468
/proc/meminfo: memFree=15890880/32872528 swapFree=17888/1798212
[pid=6366] ppid=6363 vsize=14670404 CPUtime=867.14 cores=0,2,4,6
/proc/6366/stat : 6366 (toysat) R 6363 6366 579 0 -1 4202496 3601247 0 0 0 86169 545 0 0 20 0 4 0 25990846 15022493696 3609302 33554432000 4194304 16246674 140735977887984 140735977887080 4225154 0 2 0 34082818 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6366/statm: 3667601 3609309 855 2943 0 3664495 0
[pid=6366/tid=6367] ppid=6363 vsize=14670404 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6367/stat : 6367 (toysat) S 6363 6366 579 0 -1 4202560 10 0 0 0 0 0 0 0 20 0 4 0 25990846 15022493696 3609313 33554432000 4194304 16246674 140735977887984 140625845398768 15012391 0 0 0 34082818 18446744071580610281 0 0 -1 4 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=14671428 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) S 6363 6366 579 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 4 0 25990846 15023542272 3609322 33554432000 4194304 16246674 140735977887984 140625834908912 14998865 0 0 0 34082818 18446744071580425449 0 0 -1 0 0 0 0 0 0
[pid=6366/tid=6369] ppid=6363 vsize=14671428 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6369/stat : 6369 (toysat) S 6363 6366 579 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 4 0 25990846 15023542272 3609325 33554432000 4194304 16246674 140735977887984 140625752091824 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 867.14
Current children cumulated vsize (KiB) 14670404

[startup+868.9 s]
/proc/loadavg: 1.30 1.56 1.93 2/173 6468
/proc/meminfo: memFree=15425632/32872528 swapFree=17888/1798212
[pid=6366] ppid=6363 vsize=15348292 CPUtime=868.74 cores=0,2,4,6
/proc/6366/stat : 6366 (toysat) R 6363 6366 579 0 -1 4202496 3770486 0 0 0 86305 569 0 0 20 0 4 0 25990846 15716651008 3778541 33554432000 4194304 16246674 140735977887984 140735977887080 4221016 0 2 0 34082818 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6366/statm: 3837073 3778547 855 2943 0 3833967 0
[pid=6366/tid=6367] ppid=6363 vsize=15348292 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6367/stat : 6367 (toysat) S 6363 6366 579 0 -1 4202560 10 0 0 0 0 0 0 0 20 0 4 0 25990846 15716651008 3778547 33554432000 4194304 16246674 140735977887984 140625845398768 15012391 0 0 0 34082818 18446744071580610281 0 0 -1 4 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=15348292 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) S 6363 6366 579 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 4 0 25990846 15716651008 3778547 33554432000 4194304 16246674 140735977887984 140625834908912 14998865 0 0 0 34082818 18446744071580425449 0 0 -1 0 0 0 0 0 0
[pid=6366/tid=6369] ppid=6363 vsize=15348292 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6369/stat : 6369 (toysat) S 6363 6366 579 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 4 0 25990846 15716651008 3778547 33554432000 4194304 16246674 140735977887984 140625752091824 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 868.74
Current children cumulated vsize (KiB) 15348292

[startup+869.7 s]
/proc/loadavg: 1.27 1.55 1.92 2/173 6468
/proc/meminfo: memFree=14959144/32872528 swapFree=17888/1798212
[pid=6366] ppid=6363 vsize=15686212 CPUtime=869.54 cores=0,2,4,6
/proc/6366/stat : 6366 (toysat) R 6363 6366 579 0 -1 4202496 3855049 0 0 0 86373 581 0 0 20 0 4 0 25990846 16062681088 3863104 33554432000 4194304 16246674 140735977887984 140735977887080 4225154 0 2 0 34082818 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6366/statm: 3921553 3863113 855 2943 0 3918447 0
[pid=6366/tid=6367] ppid=6363 vsize=15686212 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6367/stat : 6367 (toysat) S 6363 6366 579 0 -1 4202560 10 0 0 0 0 0 0 0 20 0 4 0 25990846 16062681088 3863120 33554432000 4194304 16246674 140735977887984 140625845398768 15012391 0 0 0 34082818 18446744071580610281 0 0 -1 4 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=15686212 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) S 6363 6366 579 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 4 0 25990846 16062681088 3863124 33554432000 4194304 16246674 140735977887984 140625834908912 14998865 0 0 0 34082818 18446744071580425449 0 0 -1 0 0 0 0 0 0
[pid=6366/tid=6369] ppid=6363 vsize=15686212 CPUtime=0 cores=0,2,4,6
/proc/6366/task/6369/stat : 6369 (toysat) S 6363 6366 579 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 4 0 25990846 16062681088 3863130 33554432000 4194304 16246674 140735977887984 140625752091824 14069300 0 0 0 34082818 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 869.54
Current children cumulated vsize (KiB) 15686212

[startup+870.1 s]
/proc/loadavg: 1.27 1.55 1.92 2/173 6468
/proc/meminfo: memFree=14959144/32872528 swapFree=17888/1798212
[pid=6366] ppid=6363 vsize=0 CPUtime=869.94 cores=0,2,4,6
/proc/6366/stat : 6366 (toysat) Z 6363 6366 579 0 -1 4202508 3864427 0 0 0 86390 604 0 0 20 0 2 0 25990846 0 0 33554432000 0 0 0 0 0 0 2 0 34082818 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6366/statm: 0 0 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=0 CPUtime=0.22 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) R 6363 6366 579 0 -1 4203588 8 0 0 0 0 22 0 0 20 0 2 0 25990846 0 0 33554432000 0 0 0 0 0 0 0 0 34082818 18446744073709551615 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 869.94
Current children cumulated vsize (KiB) 0

[startup+870.5 s]
/proc/loadavg: 1.27 1.55 1.92 2/171 6468
/proc/meminfo: memFree=21613128/32872528 swapFree=17888/1798212
[pid=6366] ppid=6363 vsize=0 CPUtime=870.34 cores=0,2,4,6
/proc/6366/stat : 6366 (toysat) Z 6363 6366 579 0 -1 4202508 3864427 0 0 0 86390 644 0 0 20 0 2 0 25990846 0 0 33554432000 0 0 0 0 0 0 2 0 34082818 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6366/statm: 0 0 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=0 CPUtime=0.62 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) R 6363 6366 579 0 -1 4203588 8 0 0 0 0 62 0 0 20 0 2 0 25990846 0 0 33554432000 0 0 0 0 0 0 0 0 34082818 18446744073709551615 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 870.34
Current children cumulated vsize (KiB) 0

[startup+870.6 s]
/proc/loadavg: 1.27 1.55 1.92 2/171 6468
/proc/meminfo: memFree=21613128/32872528 swapFree=17888/1798212
[pid=6366] ppid=6363 vsize=0 CPUtime=870.44 cores=0,2,4,6
/proc/6366/stat : 6366 (toysat) Z 6363 6366 579 0 -1 4202508 3864427 0 0 0 86390 654 0 0 20 0 2 0 25990846 0 0 33554432000 0 0 0 0 0 0 2 0 34082818 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/6366/statm: 0 0 0 0 0 0 0
[pid=6366/tid=6368] ppid=6363 vsize=0 CPUtime=0.72 cores=0,2,4,6
/proc/6366/task/6368/stat : 6368 (toysat) R 6363 6366 579 0 -1 4203588 8 0 0 0 0 72 0 0 20 0 2 0 25990846 0 0 33554432000 0 0 0 0 0 0 0 0 34082818 18446744073709551615 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 870.44
Current children cumulated vsize (KiB) 0

Child status: 1
Real time (s): 870.657
CPU time (s): 870.51
CPU user time (s): 863.905
CPU system time (s): 6.605
CPU usage (%): 99.9831
Max. virtual memory (cumulated for all children) (KiB): 15723076

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 863.905
system time used= 6.60499
maximum resident set size= 15489928
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3864427
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 865
involuntary context switches= 1105

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 3.70844 second user time and 9.12561 second system time

The end

Launcher Data 

Begin job on node123 at 2016-06-18 10:43:57
IDJOB=4103889
IDBENCH=2497
IDSOLVER=2542
FILE ID=node123/4103889-1466239435
RUNJOBID= node123-1466206081-595
PBS_JOBID= 19918154
Free space on /tmp= 71556 MiB

SOLVER NAME= toysat 2016-05-02
BENCH NAME= PB06//final/normalized-PB06/OPT-BIGINT/mps-v2-20-10/MIPLIB/miplib2003/normalized-mps-v2-20-10-momentum2.opb
COMMAND LINE= toysat +RTS -H1G -MMEMLIMITm -K1G -RTS --pb --search=bcd2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4103889-1466239435/watcher-4103889-1466239435 -o /tmp/evaluation-result-4103889-1466239435/solver-4103889-1466239435 -C 1800 -W 1900 -M 15500 --bin-var /tmp/runsolver-exch-node123-1466206081-595 --watchdog 1960  toysat +RTS -H1G -M15500m -K1G -RTS --pb --search=bcd2 HOME/instance-4103889-1466239435.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 0d80b4a073c50ac034a87a4950dc06d8
RANDOM SEED=921529891

node123.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.791
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.58
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.791
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.791
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.791
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.57
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.791
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.791
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.791
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.791
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32872528 kB
MemFree:        24388256 kB
Buffers:          118988 kB
Cached:           238404 kB
SwapCached:         1640 kB
Active:          5739288 kB
Inactive:         193940 kB
Active(anon):    5575100 kB
Inactive(anon):     2916 kB
Active(file):     164188 kB
Inactive(file):   191024 kB
Unevictable:     1798212 kB
Mlocked:           17888 kB
SwapTotal:      67108856 kB
SwapFree:       67099316 kB
Dirty:             97920 kB
Writeback:             0 kB
AnonPages:       7375532 kB
Mapped:            10492 kB
Shmem:                 0 kB
Slab:             607740 kB
SReclaimable:      51496 kB
SUnreclaim:       556244 kB
KernelStack:        1464 kB
PageTables:        18612 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545120 kB
Committed_AS:    7518944 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346412 kB
VmallocChunk:   34341988032 kB
HardwareCorrupted:     0 kB
AnonHugePages:   1794048 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7552 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 71556 MiB
End job on node123 at 2016-06-18 10:58:28