Trace number 4452220

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
FiberSCIP 2024-07-02? 3590.61 3600.38

General information on the benchmark

NamePB24/normalized-PB24/DEC-LIN/nordstrom/MIPLIB01/
dec/miplib2017/normalized-rmine11.0.u.opb
MD5SUMf2b4c33dfdd2685ad5de2b9c67a512a4
Bench CategoryDEC-LIN (decision problem, linear constraints, no UNSAT certificate)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables12292
Total number of constraints97390
Number of constraints which are clauses97367
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints23
Minimum length of a constraint2
Maximum length of a constraint12292
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 263990968
Number of bits of the biggest number in a constraint 28
Biggest sum of numbers in a constraint 763991018
Number of bits of the biggest sum of numbers30
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data

0.00/0.00	c The following solver is parallelized by UG version 1.0.0 [GitHash: cdb00f34-dirty]
0.00/0.00	c SCIP version 10.0.0 [precision: 8 byte] [memory: block] [mode: optimized] [LP solver: Soplex 7.1.0] [GitHash: 32d2b99657]
0.00/0.00	c Copyright (c) 2002-2024 Zuse Institute Berlin (ZIB)
0.00/0.00	c External libraries: 
0.00/0.00	c   Soplex 7.1.0         Linear Programming Solver developed at Zuse Institute Berlin (soplex.zib.de) [GitHash: bdec2291]
0.00/0.00	c   CppAD 20180000.0     Algorithmic Differentiation of C++ algorithms developed by B. Bell (github.com/coin-or/CppAD)
0.00/0.00	c   AMPL/MP 690e9e7      AMPL .nl file reader library (github.com/ampl/mp)
0.00/0.00	c   PaPILO 2.2.1         parallel presolve for integer and linear optimization (github.com/scipopt/papilo)
0.00/0.00	c   Nauty 2.8.8          Computing Graph Automorphism Groups by Brendan D. McKay (users.cecs.anu.edu.au/~bdm/nauty)
0.00/0.00	c   sassy 1.1            Symmetry preprocessor by Markus Anders (github.com/markusa4/sassy)
0.19/0.30	c Default LC presolving (default).
0.29/0.31	c ** Before presolving: virtualMemUsedAtLc = 247263290, getVmSize() = 171089920, SCIPgetMemUsed() = 130454650, SCIPgetMemTotal() = 141360218, SCIPgetMemExternEstim() = 105903072
0.29/0.31	c ** set memory limit for presolving in LC to 8.79602e+12 for SCIP **
35.09/35.23	c ** Estimated virtualMemUsedAtSolver = 385044944, getVmSize() = 521039872, SCIPgetMemUsed() = 292846993, SCIPgetMemTotal() = 471664345, SCIPgetMemExternEstim() = 105903072
35.09/35.23	c ** set memory limit for solvers to 4.39803e+12 for each SCIP **
35.09/35.23	c Original Problem   :
35.09/35.23	c   Problem name     : instance-4452220-1721429009
35.09/35.23	c   Variables        : 12292 (12292 binary, 0 integer, 0 implicit integer, 0 continuous)
35.09/35.23	c   Constraints      : 97390
35.09/35.23	c   Objective sense  : minimize
35.09/35.23	c Presolved Problem  :
35.09/35.23	c   Variables        : 12245 (12245 binary, 0 integer, 0 implicit integer, 0 continuous)
35.09/35.23	c   Constraints      : 97343
35.09/35.23	c Constraints        : Number
35.09/35.23	c   knapsack         : 23 
35.09/35.23	c   setppc           : 97320 
35.09/35.23	c ** Instance transfer method used: 0
35.29/35.40	c ** ParaScipInstance copy does not increase the number of variables. **
35.38/35.53	c LC is working with racing ramp-up and with rebuilding tree after racing.
35.38/35.53	c                                Nodes    Active                                            
35.38/35.53	c      Time          Nodes        Left   Solvers     Best Integer        Best Node        Gap     Best Node(S)     Gap(S)
189.15/189.75	c       190              2           1         1                -           0.0000          -
776.24/778.50	c       779              3           2         1                -           0.0000          -
839.73/842.29	c       842              4           1         1                -           0.0000          -
1080.92/1084.06	c      1084              5           2         1                -           0.0000          -
1121.71/1124.90	c      1125              6           1         1                -           0.0000          -
1619.32/1623.80	c      1624              7           2         1                -           0.0000          -
1654.63/1659.27	c      1659              8           1         1                -           0.0000          -
2533.86/2541.01	c      2541              9           2         1                -           0.0000          -
2867.96/2875.97	c      2876             10           3         1                -           0.0000          -
2880.72/2888.79	c      2889             11           2         1                -           0.0000          -
3590.12/3599.99	c      3600             11           2         0                -           0.0000          -
3590.42/3600.22	c SCIP Status        : c solving was interrupted [hard time limit reached]
3590.42/3600.22	c Total Time         : 3600.2300
3590.42/3600.22	c   solving          : 3600.2300
3590.42/3600.22	c   presolving       : 34.9260 (included in solving)
3590.42/3600.22	c B&B Tree           :
3590.42/3600.22	c   nodes (total)    : 1
3590.42/3600.22	c Solution           :
3590.42/3600.24	c   Solutions found  : 0
3590.42/3600.24	c   Primal Bound     :          -
3590.42/3600.24	c   Dual Bound       : +0.00000000000000e+00
3590.42/3600.24	c Gap                :          -
3590.52/3600.37	s UNKNOWN

Verifier Data

ERROR: no interpretation found !

Watcher Data

runsolver version 3.4.2 (svn: 4680) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 63968 MiB (61851 MiB free)
  memory of node 1: 64473 MiB (61295 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp --file-size-limit 102400 -w /tmp/evaluation-result-4452220-1721429009/watcher-4452220-1721429009 -o /tmp/evaluation-result-4452220-1721429009/solver-4452220-1721429009 -C 3600 -W 3700 -M 31000 --bin-var /tmp/runsolver-exch-nodeC023-1721423005-1028661 --watchdog 3760 ./fscip settings/racing.prm HOME/instance-4452220-1721429009.opb -qsol -t 3600 -sth 1 

running on 2 cores: 4,6

CPU governors:
  performance: 4 6

Enforcing FSIZE limit: 104857600 KiB
Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 3600 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 3630 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 3700 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Linux 4.18.0-301.1.el8.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=1032973, runsolver pid=1032970

[startup+0.100126 s]*
/proc/loadavg: 4.17 4.12 4.10 6/264 1032974
/proc/meminfo: memFree=126055272/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=68072 memory=55708 CPUtime=0.09 cores=4,6
/proc/1032973/stat : 1032973 (fscip) R 1032970 1032973 1028650 0 -1 4194304 11915 0 0 0 8 1 0 0 20 0 1 0 195065906 69705728 13927 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 17018 13927 2318 3032 0 13418 0
Current children cumulated CPU time: 0.09 s
Current children cumulated vsize: 68072 KiB
Current children cumulated memory: 55708 KiB

[startup+0.205482 s]*
/proc/loadavg: 4.17 4.12 4.10 5/264 1032974
/proc/meminfo: memFree=126010668/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=114452 memory=103356 CPUtime=0.19 cores=4,6
/proc/1032973/stat : 1032973 (fscip) R 1032970 1032973 1028650 0 -1 4194304 24112 0 0 0 16 3 0 0 20 0 1 0 195065906 117198848 25839 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 28613 25839 2542 3032 0 25013 0
Current children cumulated CPU time: 0.19 s
Current children cumulated vsize: 114452 KiB
Current children cumulated memory: 103356 KiB

[startup+0.305442 s]*
/proc/loadavg: 4.17 4.12 4.10 5/264 1032974
/proc/meminfo: memFree=125956460/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=169004 memory=155668 CPUtime=0.29 cores=4,6
/proc/1032973/stat : 1032973 (fscip) R 1032970 1032973 1028650 0 -1 4194304 37135 0 0 0 24 5 0 0 20 0 1 0 195065906 173060096 38917 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 42251 38917 2558 3032 0 38651 0
Current children cumulated CPU time: 0.29 s
Current children cumulated vsize: 169004 KiB
Current children cumulated memory: 155668 KiB

[startup+0.700246 s]
/proc/loadavg: 4.17 4.12 4.10 5/264 1032974
/proc/meminfo: memFree=125938512/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=246296 memory=232240 CPUtime=0.68 cores=4,6
/proc/1032973/stat : 1032973 (fscip) R 1032970 1032973 1028650 0 -1 4194304 54336 0 0 0 60 8 0 0 20 0 1 0 195065906 252207104 58060 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 61574 58060 2654 3032 0 57974 0
Current children cumulated CPU time: 0.68 s
Current children cumulated vsize: 246296 KiB
Current children cumulated memory: 232240 KiB

[startup+1.50023 s]
/proc/loadavg: 4.16 4.12 4.09 6/264 1032974
/proc/meminfo: memFree=125889428/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=236120 memory=220316 CPUtime=1.49 cores=4,6
/proc/1032973/stat : 1032973 (fscip) R 1032970 1032973 1028650 0 -1 4194304 68386 0 0 0 137 12 0 0 20 0 1 0 195065906 241786880 55079 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 59030 55079 2686 3032 0 55430 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 236120 KiB
Current children cumulated memory: 220316 KiB

[startup+3.10551 s]
/proc/loadavg: 4.16 4.12 4.09 5/264 1032974
/proc/meminfo: memFree=125884124/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=242204 memory=226692 CPUtime=3.08 cores=4,6
/proc/1032973/stat : 1032973 (fscip) R 1032970 1032973 1028650 0 -1 4194304 69950 0 0 0 296 12 0 0 20 0 1 0 195065906 248016896 56673 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 60551 56673 2734 3032 0 56951 0
Current children cumulated CPU time: 3.08 s
Current children cumulated vsize: 242204 KiB
Current children cumulated memory: 226692 KiB

[startup+6.30023 s]
/proc/loadavg: 4.15 4.12 4.09 5/264 1032976
/proc/meminfo: memFree=125870268/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=255944 memory=239868 CPUtime=6.27 cores=4,6
/proc/1032973/stat : 1032973 (fscip) R 1032970 1032973 1028650 0 -1 4194304 72747 0 0 0 614 13 0 0 20 0 1 0 195065906 262086656 59967 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 63986 59967 2734 3032 0 60386 0
Current children cumulated CPU time: 6.27 s
Current children cumulated vsize: 255944 KiB
Current children cumulated memory: 239868 KiB

[startup+12.7002 s]
/proc/loadavg: 4.13 4.11 4.09 6/264 1032976
/proc/meminfo: memFree=125870264/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=255944 memory=239868 CPUtime=12.65 cores=4,6
/proc/1032973/stat : 1032973 (fscip) R 1032970 1032973 1028650 0 -1 4194304 72751 0 0 0 1252 13 0 0 20 0 1 0 195065906 262086656 59967 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 63986 59967 2734 3032 0 60386 0
Current children cumulated CPU time: 12.65 s
Current children cumulated vsize: 255944 KiB
Current children cumulated memory: 239868 KiB

[startup+25.5002 s]
/proc/loadavg: 4.11 4.11 4.09 5/264 1032979
/proc/meminfo: memFree=125868828/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=256968 memory=240908 CPUtime=25.41 cores=4,6
/proc/1032973/stat : 1032973 (fscip) R 1032970 1032973 1028650 0 -1 4194304 73015 0 0 0 2528 13 0 0 20 0 1 0 195065906 263135232 60227 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 64242 60227 2734 3032 0 60642 0
Current children cumulated CPU time: 25.41 s
Current children cumulated vsize: 256968 KiB
Current children cumulated memory: 240908 KiB

[startup+51.1055 s]
/proc/loadavg: 4.07 4.10 4.09 6/266 1032986
/proc/meminfo: memFree=125423448/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=787364 memory=685368 CPUtime=50.93 cores=4,6
/proc/1032973/stat : 1032973 (fscip) S 1032970 1032973 1028650 0 -1 1077936128 252815 0 0 0 5056 37 0 0 20 0 3 0 195065906 806260736 171342 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 196841 171342 2782 3032 0 188653 0
[pid=1032973/tid=1032983] ppid=1032970 vsize=787364 memory=-7736474891429633406 CPUtime=15.64 cores=4,6
/proc/1032973/task/1032983/stat : 1032983 (fscip) R 1032970 1032973 1028650 0 -1 4194368 148199 0 0 0 1549 15 0 0 20 0 3 0 195069430 806260736 171342 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
[pid=1032973/tid=1032984] ppid=1032970 vsize=787364 memory=685368 CPUtime=0 cores=4,6
/proc/1032973/task/1032984/stat : 1032984 (fscip) S 1032970 1032973 1028650 0 -1 1077936192 29 0 0 0 0 0 0 0 20 0 3 0 195069460 806260736 171342 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
Current children cumulated CPU time: 50.93 s
Current children cumulated vsize: 787364 KiB
Current children cumulated memory: 685368 KiB

[startup+102.3 s]
/proc/loadavg: 4.48 4.19 4.12 6/263 1032994
/proc/meminfo: memFree=125108760/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=1112628 memory=986184 CPUtime=101.99 cores=4,6
/proc/1032973/stat : 1032973 (fscip) S 1032970 1032973 1028650 0 -1 1077936128 370055 0 0 0 10147 52 0 0 20 0 3 0 195065906 1139331072 246546 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 278157 246546 2910 3032 0 272314 0
[pid=1032973/tid=1032983] ppid=1032970 vsize=1112628 memory=983976 CPUtime=66.7 cores=4,6
/proc/1032973/task/1032983/stat : 1032983 (fscip) R 1032970 1032973 1028650 0 -1 4194368 265408 0 0 0 6640 30 0 0 20 0 3 0 195069430 1139331072 246546 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
[pid=1032973/tid=1032984] ppid=1032970 vsize=1112628 memory=983620 CPUtime=0 cores=4,6
/proc/1032973/task/1032984/stat : 1032984 (fscip) S 1032970 1032973 1028650 0 -1 1077936192 60 0 0 0 0 0 0 0 20 0 3 0 195069460 1139331072 246546 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
Current children cumulated CPU time: 101.99 s
Current children cumulated vsize: 1112628 KiB
Current children cumulated memory: 986184 KiB

[startup+162.3 s]
/proc/loadavg: 4.33 4.20 4.13 6/263 1033004
/proc/meminfo: memFree=125047352/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=1112628 memory=986904 CPUtime=161.83 cores=4,6
/proc/1032973/stat : 1032973 (fscip) S 1032970 1032973 1028650 0 -1 1077936128 388004 0 0 0 16129 54 0 0 20 0 3 0 195065906 1139331072 246726 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 278157 246726 2910 3032 0 272973 0
[pid=1032973/tid=1032983] ppid=1032970 vsize=1112628 memory=986652 CPUtime=126.52 cores=4,6
/proc/1032973/task/1032983/stat : 1032983 (fscip) R 1032970 1032973 1028650 0 -1 4194368 283339 0 0 0 12621 31 0 0 20 0 3 0 195069430 1139331072 246726 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
[pid=1032973/tid=1032984] ppid=1032970 vsize=1112628 memory=986904 CPUtime=0 cores=4,6
/proc/1032973/task/1032984/stat : 1032984 (fscip) S 1032970 1032973 1028650 0 -1 1077936192 78 0 0 0 0 0 0 0 20 0 3 0 195069460 1139331072 246726 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
Current children cumulated CPU time: 161.83 s
Current children cumulated vsize: 1112628 KiB
Current children cumulated memory: 986904 KiB

[startup+222.3 s]
/proc/loadavg: 4.22 4.19 4.13 6/262 1033014
/proc/meminfo: memFree=125016504/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=1178164 memory=999952 CPUtime=221.65 cores=4,6
/proc/1032973/stat : 1032973 (fscip) S 1032970 1032973 1028650 0 -1 1077936128 421893 0 0 0 22108 57 0 0 20 0 3 0 195065906 1206439936 249988 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 294541 249988 2910 3032 0 276046 0
[pid=1032973/tid=1032983] ppid=1032970 vsize=1178164 memory=987240 CPUtime=186.35 cores=4,6
/proc/1032973/task/1032983/stat : 1032983 (fscip) R 1032970 1032973 1028650 0 -1 4194368 317187 0 0 0 18601 34 0 0 20 0 3 0 195069430 1206439936 249988 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
[pid=1032973/tid=1032984] ppid=1032970 vsize=1178164 memory=994796 CPUtime=0 cores=4,6
/proc/1032973/task/1032984/stat : 1032984 (fscip) S 1032970 1032973 1028650 0 -1 1077936192 86 0 0 0 0 0 0 0 20 0 3 0 195069460 1206439936 249988 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
Current children cumulated CPU time: 221.65 s
Current children cumulated vsize: 1178164 KiB
Current children cumulated memory: 999952 KiB

[startup+282.3 s]
/proc/loadavg: 4.08 4.16 4.11 6/263 1033027
/proc/meminfo: memFree=125016092/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=1178164 memory=999952 CPUtime=281.49 cores=4,6
/proc/1032973/stat : 1032973 (fscip) S 1032970 1032973 1028650 0 -1 1077936128 441065 0 0 0 28091 58 0 0 20 0 3 0 195065906 1206439936 249988 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 294541 249988 2910 3032 0 276046 0
[pid=1032973/tid=1032983] ppid=1032970 vsize=1178164 memory=994796 CPUtime=246.19 cores=4,6
/proc/1032973/task/1032983/stat : 1032983 (fscip) R 1032970 1032973 1028650 0 -1 4194368 336350 0 0 0 24584 35 0 0 20 0 3 0 195069430 1206439936 249988 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
[pid=1032973/tid=1032984] ppid=1032970 vsize=1178164 memory=999952 CPUtime=0 cores=4,6
/proc/1032973/task/1032984/stat : 1032984 (fscip) S 1032970 1032973 1028650 0 -1 1077936192 95 0 0 0 0 0 0 0 20 0 3 0 195069460 1206439936 249988 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
Current children cumulated CPU time: 281.49 s
Current children cumulated vsize: 1178164 KiB
Current children cumulated memory: 999952 KiB

[startup+342.3 s]
/proc/loadavg: 4.13 4.16 4.11 6/263 1033037
/proc/meminfo: memFree=124992736/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=1178164 memory=999952 CPUtime=341.34 cores=4,6
/proc/1032973/stat : 1032973 (fscip) S 1032970 1032973 1028650 0 -1 1077936128 452161 0 0 0 34075 59 0 0 20 0 3 0 195065906 1206439936 249988 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 294541 249988 2910 3032 0 276046 0

################
# More data... #
################

[pid=1032973] ppid=1032970 vsize=1182548 memory=1035476 CPUtime=3452.89 cores=4,6
/proc/1032973/stat : 1032973 (fscip) S 1032970 1032973 1028650 0 -1 1077936128 735200 0 0 0 345202 87 0 0 20 0 3 0 195065906 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 295637 258869 2974 3032 0 280549 0
[pid=1032973/tid=1032983] ppid=1032970 vsize=1182548 memory=11648 CPUtime=3417.54 cores=4,6
/proc/1032973/task/1032983/stat : 1032983 (fscip) R 1032970 1032973 1028650 0 -1 4194368 629846 0 0 0 341693 61 0 0 20 0 3 0 195069430 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
[pid=1032973/tid=1032984] ppid=1032970 vsize=1182548 memory=1035476 CPUtime=0.04 cores=4,6
/proc/1032973/task/1032984/stat : 1032984 (fscip) S 1032970 1032973 1028650 0 -1 1077936192 527 0 0 0 1 3 0 0 20 0 3 0 195069460 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
Current children cumulated CPU time: 3452.89 s
Current children cumulated vsize: 1182548 KiB
Current children cumulated memory: 1035476 KiB

[startup+3522.3 s]
/proc/loadavg: 4.15 4.12 4.09 5/265 1035174
/proc/meminfo: memFree=124890272/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=1182548 memory=1035476 CPUtime=3512.74 cores=4,6
/proc/1032973/stat : 1032973 (fscip) S 1032970 1032973 1028650 0 -1 1077936128 735381 0 0 0 351186 88 0 0 20 0 3 0 195065906 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 295637 258869 2974 3032 0 280549 0
[pid=1032973/tid=1032983] ppid=1032970 vsize=1182548 memory=12952 CPUtime=3477.39 cores=4,6
/proc/1032973/task/1032983/stat : 1032983 (fscip) R 1032970 1032973 1028650 0 -1 4194368 630017 0 0 0 347678 61 0 0 20 0 3 0 195069430 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
[pid=1032973/tid=1032984] ppid=1032970 vsize=1182548 memory=1035476 CPUtime=0.05 cores=4,6
/proc/1032973/task/1032984/stat : 1032984 (fscip) S 1032970 1032973 1028650 0 -1 1077936192 537 0 0 0 1 4 0 0 20 0 3 0 195069460 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
Current children cumulated CPU time: 3512.74 s
Current children cumulated vsize: 1182548 KiB
Current children cumulated memory: 1035476 KiB

[startup+3582.3 s]
/proc/loadavg: 4.11 4.11 4.09 5/265 1035185
/proc/meminfo: memFree=124886984/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=1182548 memory=1035476 CPUtime=3572.58 cores=4,6
/proc/1032973/stat : 1032973 (fscip) S 1032970 1032973 1028650 0 -1 1077936128 735602 0 0 0 357170 88 0 0 20 0 3 0 195065906 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 295637 258869 2974 3032 0 280549 0
[pid=1032973/tid=1032983] ppid=1032970 vsize=1182548 memory=15996 CPUtime=3537.22 cores=4,6
/proc/1032973/task/1032983/stat : 1032983 (fscip) R 1032970 1032973 1028650 0 -1 4194368 630232 0 0 0 353661 61 0 0 20 0 3 0 195069430 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
[pid=1032973/tid=1032984] ppid=1032970 vsize=1182548 memory=1035476 CPUtime=0.05 cores=4,6
/proc/1032973/task/1032984/stat : 1032984 (fscip) S 1032970 1032973 1028650 0 -1 1077936192 543 0 0 0 1 4 0 0 20 0 3 0 195069460 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
Current children cumulated CPU time: 3572.58 s
Current children cumulated vsize: 1182548 KiB
Current children cumulated memory: 1035476 KiB

[startup+3600.38 s]
# the end of solver process 1032973 was just reported to runsolver
# Child status: 0

# cumulated CPU time of all completed processes:  user=3589.71 s, system=0.895441 s

Solver just ended.
??? end of watcher thread
Dumping a history of the last processes samples

[startup+3584 s]*
/proc/loadavg: 4.11 4.11 4.09 5/265 1035187
/proc/meminfo: memFree=124887016/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=1182548 memory=1035476 CPUtime=3574.28 cores=4,6
/proc/1032973/stat : 1032973 (fscip) S 1032970 1032973 1028650 0 -1 1077936128 735602 0 0 0 357340 88 0 0 20 0 3 0 195065906 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 295637 258869 2974 3032 0 280549 0
[pid=1032973/tid=1032983] ppid=1032970 vsize=1182548 memory=-7312575608146203752 CPUtime=3538.92 cores=4,6
/proc/1032973/task/1032983/stat : 1032983 (fscip) R 1032970 1032973 1028650 0 -1 4194368 630232 0 0 0 353831 61 0 0 20 0 3 0 195069430 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
[pid=1032973/tid=1032984] ppid=1032970 vsize=1182548 memory=100 CPUtime=0.05 cores=4,6
/proc/1032973/task/1032984/stat : 1032984 (fscip) S 1032970 1032973 1028650 0 -1 1077936192 543 0 0 0 1 4 0 0 20 0 3 0 195069460 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
Current children cumulated CPU time: 3574.28 s
Current children cumulated vsize: 1182548 KiB
Current children cumulated memory: 1035476 KiB

[startup+3590.4 s]
/proc/loadavg: 4.10 4.11 4.09 5/265 1035187
/proc/meminfo: memFree=124887028/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=1182548 memory=1035476 CPUtime=3580.65 cores=4,6
/proc/1032973/stat : 1032973 (fscip) S 1032970 1032973 1028650 0 -1 1077936128 735626 0 0 0 357977 88 0 0 20 0 3 0 195065906 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 295637 258869 2974 3032 0 280549 0
[pid=1032973/tid=1032983] ppid=1032970 vsize=1182548 memory=5432 CPUtime=3545.3 cores=4,6
/proc/1032973/task/1032983/stat : 1032983 (fscip) R 1032970 1032973 1028650 0 -1 4194368 630254 0 0 0 354469 61 0 0 20 0 3 0 195069430 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
[pid=1032973/tid=1032984] ppid=1032970 vsize=1182548 memory=1035476 CPUtime=0.05 cores=4,6
/proc/1032973/task/1032984/stat : 1032984 (fscip) S 1032970 1032973 1028650 0 -1 1077936192 545 0 0 0 1 4 0 0 20 0 3 0 195069460 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
Current children cumulated CPU time: 3580.65 s
Current children cumulated vsize: 1182548 KiB
Current children cumulated memory: 1035476 KiB

[startup+3596.8 s]
/proc/loadavg: 4.09 4.11 4.09 5/265 1035189
/proc/meminfo: memFree=124887452/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=1182548 memory=1035476 CPUtime=3587.03 cores=4,6
/proc/1032973/stat : 1032973 (fscip) S 1032970 1032973 1028650 0 -1 1077936128 735633 0 0 0 358615 88 0 0 20 0 3 0 195065906 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 295637 258869 2974 3032 0 280549 0
[pid=1032973/tid=1032983] ppid=1032970 vsize=1182548 memory=15996 CPUtime=3551.68 cores=4,6
/proc/1032973/task/1032983/stat : 1032983 (fscip) R 1032970 1032973 1028650 0 -1 4194368 630261 0 0 0 355107 61 0 0 20 0 3 0 195069430 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
[pid=1032973/tid=1032984] ppid=1032970 vsize=1182548 memory=2280 CPUtime=0.05 cores=4,6
/proc/1032973/task/1032984/stat : 1032984 (fscip) S 1032970 1032973 1028650 0 -1 1077936192 545 0 0 0 1 4 0 0 20 0 3 0 195069460 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
Current children cumulated CPU time: 3587.03 s
Current children cumulated vsize: 1182548 KiB
Current children cumulated memory: 1035476 KiB

[startup+3598.4 s]
/proc/loadavg: 4.08 4.11 4.09 5/266 1035190
/proc/meminfo: memFree=124887348/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=1182548 memory=1035476 CPUtime=3588.63 cores=4,6
/proc/1032973/stat : 1032973 (fscip) S 1032970 1032973 1028650 0 -1 1077936128 735633 0 0 0 358775 88 0 0 20 0 3 0 195065906 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 295637 258869 2974 3032 0 280549 0
[pid=1032973/tid=1032983] ppid=1032970 vsize=1182548 memory=2204 CPUtime=3553.27 cores=4,6
/proc/1032973/task/1032983/stat : 1032983 (fscip) R 1032970 1032973 1028650 0 -1 4194368 630261 0 0 0 355266 61 0 0 20 0 3 0 195069430 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
[pid=1032973/tid=1032984] ppid=1032970 vsize=1182548 memory=5544 CPUtime=0.05 cores=4,6
/proc/1032973/task/1032984/stat : 1032984 (fscip) S 1032970 1032973 1028650 0 -1 1077936192 545 0 0 0 1 4 0 0 20 0 3 0 195069460 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
Current children cumulated CPU time: 3588.63 s
Current children cumulated vsize: 1182548 KiB
Current children cumulated memory: 1035476 KiB

[startup+3599.2 s]
/proc/loadavg: 4.08 4.11 4.09 5/266 1035190
/proc/meminfo: memFree=124887348/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=1182548 memory=1035476 CPUtime=3589.43 cores=4,6
/proc/1032973/stat : 1032973 (fscip) S 1032970 1032973 1028650 0 -1 1077936128 735633 0 0 0 358855 88 0 0 20 0 3 0 195065906 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 295637 258869 2974 3032 0 280549 0
[pid=1032973/tid=1032983] ppid=1032970 vsize=1182548 memory=11648 CPUtime=3554.07 cores=4,6
/proc/1032973/task/1032983/stat : 1032983 (fscip) R 1032970 1032973 1028650 0 -1 4194368 630261 0 0 0 355346 61 0 0 20 0 3 0 195069430 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
[pid=1032973/tid=1032984] ppid=1032970 vsize=1182548 memory=1035476 CPUtime=0.05 cores=4,6
/proc/1032973/task/1032984/stat : 1032984 (fscip) S 1032970 1032973 1028650 0 -1 1077936192 545 0 0 0 1 4 0 0 20 0 3 0 195069460 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
Current children cumulated CPU time: 3589.43 s
Current children cumulated vsize: 1182548 KiB
Current children cumulated memory: 1035476 KiB

[startup+3600 s]*
/proc/loadavg: 4.08 4.11 4.09 5/266 1035190
/proc/meminfo: memFree=124887388/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=1182548 memory=1035476 CPUtime=3590.23 cores=4,6
/proc/1032973/stat : 1032973 (fscip) S 1032970 1032973 1028650 0 -1 1077936128 736009 0 0 0 358935 88 0 0 20 0 3 0 195065906 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 295637 258869 2974 3032 0 280549 0
[pid=1032973/tid=1032983] ppid=1032970 vsize=1182548 memory=35124 CPUtime=3554.88 cores=4,6
/proc/1032973/task/1032983/stat : 1032983 (fscip) R 1032970 1032973 1028650 0 -1 4194368 630639 0 0 0 355427 61 0 0 20 0 3 0 195069430 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
[pid=1032973/tid=1032984] ppid=1032970 vsize=1182548 memory=-6933350163988323774 CPUtime=0.05 cores=4,6
/proc/1032973/task/1032984/stat : 1032984 (fscip) S 1032970 1032973 1028650 0 -1 1077936192 545 0 0 0 1 4 0 0 20 0 3 0 195069460 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
Current children cumulated CPU time: 3590.23 s
Current children cumulated vsize: 1182548 KiB
Current children cumulated memory: 1035476 KiB

[startup+3600.2 s]
/proc/loadavg: 4.08 4.11 4.09 5/266 1035190
/proc/meminfo: memFree=124887388/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=1182548 memory=1035476 CPUtime=3590.42 cores=4,6
/proc/1032973/stat : 1032973 (fscip) S 1032970 1032973 1028650 0 -1 1077936128 736993 0 0 0 358954 88 0 0 20 0 3 0 195065906 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 295637 258869 2974 3032 0 280549 0
[pid=1032973/tid=1032983] ppid=1032970 vsize=1182548 memory=5480 CPUtime=3555.07 cores=4,6
/proc/1032973/task/1032983/stat : 1032983 (fscip) R 1032970 1032973 1028650 0 -1 4194368 631605 0 0 0 355446 61 0 0 20 0 3 0 195069430 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
[pid=1032973/tid=1032984] ppid=1032970 vsize=1182548 memory=2784 CPUtime=0.05 cores=4,6
/proc/1032973/task/1032984/stat : 1032984 (fscip) S 1032970 1032973 1028650 0 -1 1077936192 545 0 0 0 1 4 0 0 20 0 3 0 195069460 1210929152 258869 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
Current children cumulated CPU time: 3590.42 s
Current children cumulated vsize: 1182548 KiB
Current children cumulated memory: 1035476 KiB

[startup+3600.3 s]
/proc/loadavg: 4.08 4.11 4.09 5/266 1035190
/proc/meminfo: memFree=124887388/131524600 swapFree=131070704/131071996
[pid=1032973] ppid=1032970 vsize=917440 memory=809540 CPUtime=3590.52 cores=4,6
/proc/1032973/stat : 1032973 (fscip) R 1032970 1032973 1028650 0 -1 4194304 739857 0 0 0 358964 88 0 0 20 0 2 0 195065906 939458560 202385 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
/proc/1032973/statm: 229360 202385 2974 3032 0 224663 0
[pid=1032973/tid=1032984] ppid=1032970 vsize=917440 memory=1035476 CPUtime=0.05 cores=4,6
/proc/1032973/task/1032984/stat : 1032984 (fscip) S 1032970 1032973 1028650 0 -1 1077936192 545 0 0 0 1 4 0 0 20 0 2 0 195069460 939458560 202385 134679101440 4194304 16613245 140730254478912 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18822208 18930672 45252608 140730254484499 140730254484615 140730254484615 140730254491632 0
Current children cumulated CPU time: 3590.52 s
Current children cumulated vsize: 917440 KiB
Current children cumulated memory: 809540 KiB

Child status: 0

Real time (s): 3600.38
CPU time (s): 3590.61
CPU user time (s): 3589.71
CPU system time (s): 0.895441
CPU usage (%): 99.7286
Max. virtual memory (cumulated for all children) (KiB): 1182548
Max. memory (cumulated for all children) (KiB): 1035476

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 3589.71
system time used= 0.895441
maximum resident set size= 1035476
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 746431
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3609
involuntary context switches= 1357


# summary of solver processes directly reported to runsolver:
#   pid: 1032973
#   total CPU time (s): 3590.61
#   total CPU user time (s): 3589.71
#   total CPU system time (s): 0.895441

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 7.00685 second user time and 16.8334 second system time

The end

Launcher Data

Begin job on nodeC023 at 2024-07-20 00:43:42
IDJOB=4452220
IDBENCH=164078
IDSOLVER=3064
FILE ID=nodeC023/4452220-1721429009
RUNJOBID= nodeC023-1721423005-1028661
SLURM_JOB_ID= 7545608
Free space on /tmp= 889992 MiB

SOLVER NAME= FiberSCIP 2024-07-02
BENCH NAME= PB24/normalized-PB24/DEC-LIN/nordstrom/MIPLIB01/dec/miplib2017/normalized-rmine11.0.u.opb
COMMAND LINE= ./fscip settings/racing.prm BENCHNAME -qsol -t TIMEOUT -sth 1
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp --file-size-limit 102400 -w /tmp/evaluation-result-4452220-1721429009/watcher-4452220-1721429009 -o /tmp/evaluation-result-4452220-1721429009/solver-4452220-1721429009 -C 3600 -W 3700 -M 31000 --bin-var /tmp/runsolver-exch-nodeC023-1721423005-1028661 --watchdog 3760  ./fscip settings/racing.prm HOME/instance-4452220-1721429009.opb -qsol -t 3600 -sth 1

TIME LIMIT= 3600 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 2

MD5SUM BENCH= f2b4c33dfdd2685ad5de2b9c67a512a4
RANDOM SEED=847797306

nodeC023.alineos.net Linux 4.18.0-301.1.el8.x86_64 #1 SMP Tue Apr 13 16:24:22 UTC 2021

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3665.364
cache size	: 15360 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 6999.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3689.053
cache size	: 15360 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 7005.48
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3600.190
cache size	: 15360 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 6999.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3600.062
cache size	: 15360 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 7005.48
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3670.113
cache size	: 15360 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 6999.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3689.084
cache size	: 15360 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 7005.48
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3654.900
cache size	: 15360 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 6999.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3600.102
cache size	: 15360 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 7005.48
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       131524600 kB
MemFree:        126102768 kB
MemAvailable:   128289276 kB
Buffers:          565452 kB
Cached:          2457348 kB
SwapCached:           80 kB
Active:          1198232 kB
Inactive:        3652832 kB
Active(anon):       1580 kB
Inactive(anon):  1872452 kB
Active(file):    1196652 kB
Inactive(file):  1780380 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:      131071996 kB
SwapFree:       131070704 kB
Dirty:             31196 kB
Writeback:            12 kB
AnonPages:       1828232 kB
Mapped:           143116 kB
Shmem:             45728 kB
KReclaimable:      98596 kB
Slab:             192452 kB
SReclaimable:      98596 kB
SUnreclaim:        93856 kB
KernelStack:        4208 kB
PageTables:        13024 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    196834296 kB
Committed_AS:    1178576 kB
VmallocTotal:   34359738367 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
Percpu:            33760 kB
HardwareCorrupted:     0 kB
AnonHugePages:   1574912 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
FileHugePages:         0 kB
FilePmdMapped:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB
DirectMap4k:      316116 kB
DirectMap2M:    38254592 kB
DirectMap1G:    97517568 kB

Free space on /tmp at the end= 889968 MiB
End job on nodeC023 at 2024-07-20 01:43:43