Trace number 4464184

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
FiberSCIP 2024-07-30? (TO) 3600.15 3610.27

General information on the benchmark

NamePB24/normalized-PB11/OPT-LIN/
kullmann/normalized-mul_8_14_shg.opb
MD5SUMc0e25a71d8726d42148f3e780c01ff44
Bench CategoryOPT-LIN (optimization problem, linear constraints, no OPT/UNSAT certificate)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark56
Best CPU time to get the best result obtained on this benchmark3558.02
Has Objective FunctionYES
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables14300
Total number of constraints65280
Number of constraints which are clauses65280
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint60
Maximum length of a constraint93
Number of terms in the objective function 14300
Biggest coefficient in the objective function 1
Number of bits for the biggest coefficient in the objective function 1
Sum of the numbers in the objective function 14300
Number of bits of the sum of numbers in the objective function 14
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 14300
Number of bits of the biggest sum of numbers14
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c The following solver is parallelized by UG version 1.0.0 [GitHash: e6e2b6ff-dirty]
0.00/0.00	c SCIP version 10.0.0 [precision: 8 byte] [memory: block] [mode: optimized] [LP solver: Soplex 7.1.1] [GitHash: 020d91813d]
0.00/0.00	c Copyright (c) 2002-2024 Zuse Institute Berlin (ZIB)
0.00/0.00	c External libraries: 
0.00/0.00	c   Soplex 7.1.1         Linear Programming Solver developed at Zuse Institute Berlin (soplex.zib.de) [GitHash: b4905e82]
0.00/0.00	c   CppAD 20180000.0     Algorithmic Differentiation of C++ algorithms developed by B. Bell (github.com/coin-or/CppAD)
0.00/0.00	c   AMPL/MP 690e9e7      AMPL .nl file reader library (github.com/ampl/mp)
0.00/0.00	c   PaPILO 2.4.0         parallel presolve for integer and linear optimization (github.com/scipopt/papilo)
0.00/0.00	c   Nauty 2.8.8          Computing Graph Automorphism Groups by Brendan D. McKay (users.cecs.anu.edu.au/~bdm/nauty)
0.00/0.00	c   sassy 1.1            Symmetry preprocessor by Markus Anders (github.com/markusa4/sassy)
2.68/2.73	c Default LC presolving (default).
2.68/2.73	c ** Before presolving: virtualMemUsedAtLc = 988159393, getVmSize() = 768761856, SCIPgetMemUsed() = 527426553, SCIPgetMemTotal() = 730975913, SCIPgetMemExternEstim() = 257183480
2.68/2.73	c ** set memory limit for presolving in LC to 8.7958e+12 for SCIP **
21.05/21.16	c ** Estimated virtualMemUsedAtSolver = 2856490482, getVmSize() = 4290281472, SCIPgetMemUsed() = 2312061859, SCIPgetMemTotal() = 4027552243, SCIPgetMemExternEstim() = 257183480
21.05/21.16	c ** set memory limit for solvers to 4.39796e+12 for each SCIP **
21.05/21.16	c Original Problem   :
21.05/21.16	c   Problem name     : instance-4464184-1722382202
21.05/21.16	c   Variables        : 14300 (14300 binary, 0 integer, 0 implicit integer, 0 continuous)
21.05/21.16	c   Constraints      : 65280
21.05/21.16	c   Objective sense  : minimize
21.05/21.16	c Presolved Problem  :
21.05/21.16	c   Variables        : 14300 (14300 binary, 0 integer, 0 implicit integer, 0 continuous)
21.05/21.16	c   Constraints      : 65280
21.05/21.16	c Constraints        : Number
21.05/21.16	c   logicor          : 65280 
21.05/21.16	c ** Instance transfer method used: 0
21.43/21.58	c ** ParaScipInstance copy does not increase the number of variables. **
22.12/22.23	c LC is working with racing ramp-up and with rebuilding tree after racing.
22.12/22.23	c                                Nodes    Active                                            
22.12/22.23	c      Time          Nodes        Left   Solvers     Best Integer        Best Node        Gap     Best Node(S)     Gap(S)
33.80/33.95	c *      34              0           1         1          68.0000                -          -
33.80/33.95	c        34              0           1         1          68.0000                -          -

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver version 3.4.2 (svn: 4680) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 64096 MiB (62416 MiB free)
  memory of node 1: 64473 MiB (63730 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp --file-size-limit 102400 -w /tmp/evaluation-result-4464184-1722382202/watcher-4464184-1722382202 -o /tmp/evaluation-result-4464184-1722382202/solver-4464184-1722382202 -C 3600 -W 3700 -M 31000 --bin-var /tmp/runsolver-exch-nodeC007-1722378594-28206 --watchdog 3760 ./fscip settings/racing.prm HOME/instance-4464184-1722382202.opb -qsol -t 3600 -sth 1 

running on 2 cores: 4,6

CPU governors:
  performance: 4 6

Enforcing FSIZE limit: 104857600 KiB
Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 3600 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 3630 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 3700 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Linux 4.18.0-301.1.el8.x86_64
Starting watchdog thread
solver pid=30637, runsolver pid=30634
Current StackSize limit: 8192 KiB


[startup+0.100087 s]*
/proc/loadavg: 4.16 4.23 4.10 5/249 30638
/proc/meminfo: memFree=129153508/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=42348 memory=30572 CPUtime=0.09 cores=4,6
/proc/30637/stat : 30637 (fscip) R 30634 30637 28195 0 -1 4194304 5547 0 0 0 8 1 0 0 20 0 1 0 4590847 43364352 7643 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 10587 7643 2347 3035 0 6983 0
Current children cumulated CPU time: 0.09 s
Current children cumulated vsize: 42348 KiB
Current children cumulated memory: 30572 KiB

[startup+0.20735 s]*
/proc/loadavg: 4.16 4.23 4.10 5/249 30638
/proc/meminfo: memFree=129147380/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=48084 memory=36376 CPUtime=0.2 cores=4,6
/proc/30637/stat : 30637 (fscip) R 30634 30637 28195 0 -1 4194304 6963 0 0 0 18 2 0 0 20 0 1 0 4590847 49238016 9094 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 12021 9094 2347 3035 0 8417 0
Current children cumulated CPU time: 0.2 s
Current children cumulated vsize: 48084 KiB
Current children cumulated memory: 36376 KiB

[startup+0.306613 s]*
/proc/loadavg: 4.16 4.23 4.10 5/249 30638
/proc/meminfo: memFree=129143348/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=52536 memory=40860 CPUtime=0.3 cores=4,6
/proc/30637/stat : 30637 (fscip) R 30634 30637 28195 0 -1 4194304 8098 0 0 0 28 2 0 0 20 0 1 0 4590847 53796864 10215 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 13134 10215 2347 3035 0 9530 0
Current children cumulated CPU time: 0.3 s
Current children cumulated vsize: 52536 KiB
Current children cumulated memory: 40860 KiB

[startup+0.700278 s]
/proc/loadavg: 4.16 4.23 4.10 5/249 30638
/proc/meminfo: memFree=129132820/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=77040 memory=65412 CPUtime=0.69 cores=4,6
/proc/30637/stat : 30637 (fscip) R 30634 30637 28195 0 -1 4194304 14247 0 0 0 65 4 0 0 20 0 1 0 4590847 78888960 16353 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 19260 16353 2347 3035 0 15656 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 77040 KiB
Current children cumulated memory: 65412 KiB

[startup+1.5003 s]
/proc/loadavg: 4.15 4.23 4.10 5/249 30638
/proc/meminfo: memFree=129100508/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=137808 memory=125848 CPUtime=1.49 cores=4,6
/proc/30637/stat : 30637 (fscip) R 30634 30637 28195 0 -1 4194304 29338 0 0 0 142 7 0 0 20 0 1 0 4590847 141115392 31462 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 34452 31462 2347 3035 0 30848 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 137808 KiB
Current children cumulated memory: 125848 KiB

[startup+3.11298 s]
/proc/loadavg: 4.15 4.23 4.10 5/247 30684
/proc/meminfo: memFree=128152452/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=750744 memory=735520 CPUtime=3.1 cores=4,6
/proc/30637/stat : 30637 (fscip) R 30634 30637 28195 0 -1 4194304 193842 0 0 0 279 31 0 0 20 0 1 0 4590847 768761856 183880 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 187686 183880 2598 3035 0 184082 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 750744 KiB
Current children cumulated memory: 735520 KiB

[startup+6.30029 s]
/proc/loadavg: 4.14 4.22 4.10 5/247 30684
/proc/meminfo: memFree=126550084/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=996288 memory=967560 CPUtime=6.27 cores=4,6
/proc/30637/stat : 30637 (fscip) R 30634 30637 28195 0 -1 4194304 235003 0 0 0 566 61 0 0 20 0 1 0 4590847 1020198912 241890 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 249072 241890 2710 3035 0 245468 0
Current children cumulated CPU time: 6.27 s
Current children cumulated vsize: 996288 KiB
Current children cumulated memory: 967560 KiB

[startup+12.7003 s]
/proc/loadavg: 4.12 4.22 4.10 5/247 30742
/proc/meminfo: memFree=124787952/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=2762648 memory=2378612 CPUtime=12.66 cores=4,6
/proc/30637/stat : 30637 (fscip) R 30634 30637 28195 0 -1 4194304 313251 0 0 0 1166 100 0 0 20 0 1 0 4590847 2828951552 594653 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 690662 594653 2758 3035 0 687058 0
Current children cumulated CPU time: 12.66 s
Current children cumulated vsize: 2762648 KiB
Current children cumulated memory: 2378612 KiB

[startup+25.5004 s]
/proc/loadavg: 4.18 4.23 4.10 5/249 30745
/proc/meminfo: memFree=122334756/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=5408912 memory=4877584 CPUtime=25.42 cores=4,6
/proc/30637/stat : 30637 (fscip) S 30634 30637 28195 0 -1 1077936128 590789 0 0 0 2373 169 0 0 20 0 3 0 4590847 5538725888 1219396 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 1352228 1219396 2790 3035 0 1343830 0
[pid=30637/tid=30744] ppid=30634 vsize=5408912 memory=11516 CPUtime=3.89 cores=4,6
/proc/30637/task/30744/stat : 30744 (fscip) R 30634 30637 28195 0 -1 4194368 193029 0 0 0 354 35 0 0 20 0 3 0 4592964 5538725888 1219396 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
[pid=30637/tid=30745] ppid=30634 vsize=5408912 memory=3665788 CPUtime=0 cores=4,6
/proc/30637/task/30745/stat : 30745 (fscip) S 30634 30637 28195 0 -1 1077936192 0 0 0 0 0 0 0 0 20 0 3 0 4593070 5538725888 1219396 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
Current children cumulated CPU time: 25.42 s
Current children cumulated vsize: 5408912 KiB
Current children cumulated memory: 4877584 KiB

[startup+51.1134 s]
/proc/loadavg: 4.23 4.24 4.11 5/248 30750
/proc/meminfo: memFree=121636172/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=5932008 memory=5385376 CPUtime=50.97 cores=4,6
/proc/30637/stat : 30637 (fscip) S 30634 30637 28195 0 -1 1077936128 747729 0 0 0 4871 226 0 0 20 0 3 0 4590847 6074376192 1346344 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 1483002 1346344 2966 3035 0 1476658 0
[pid=30637/tid=30744] ppid=30634 vsize=5932008 memory=5385376 CPUtime=29.44 cores=4,6
/proc/30637/task/30744/stat : 30744 (fscip) R 30634 30637 28195 0 -1 4194368 348086 0 0 0 2851 93 0 0 20 0 3 0 4592964 6074376192 1346344 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
[pid=30637/tid=30745] ppid=30634 vsize=5932008 memory=5385376 CPUtime=0 cores=4,6
/proc/30637/task/30745/stat : 30745 (fscip) S 30634 30637 28195 0 -1 1077936192 6 0 0 0 0 0 0 0 20 0 3 0 4593070 6074376192 1346344 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
Current children cumulated CPU time: 50.97 s
Current children cumulated vsize: 5932008 KiB
Current children cumulated memory: 5385376 KiB

[startup+102.3 s]
/proc/loadavg: 4.22 4.23 4.11 5/248 30758
/proc/meminfo: memFree=121589536/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=5932008 memory=5385440 CPUtime=102.03 cores=4,6
/proc/30637/stat : 30637 (fscip) S 30634 30637 28195 0 -1 1077936128 755029 0 0 0 9975 228 0 0 20 0 3 0 4590847 6074376192 1346360 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 1483002 1346360 2982 3035 0 1476658 0
[pid=30637/tid=30744] ppid=30634 vsize=5932008 memory=5385440 CPUtime=80.49 cores=4,6
/proc/30637/task/30744/stat : 30744 (fscip) R 30634 30637 28195 0 -1 4194368 355372 0 0 0 7955 94 0 0 20 0 3 0 4592964 6074376192 1346360 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
[pid=30637/tid=30745] ppid=30634 vsize=5932008 memory=12624 CPUtime=0 cores=4,6
/proc/30637/task/30745/stat : 30745 (fscip) S 30634 30637 28195 0 -1 1077936192 20 0 0 0 0 0 0 0 20 0 3 0 4593070 6074376192 1346360 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
Current children cumulated CPU time: 102.03 s
Current children cumulated vsize: 5932008 KiB
Current children cumulated memory: 5385440 KiB

[startup+162.3 s]
/proc/loadavg: 4.14 4.20 4.10 5/249 30771
/proc/meminfo: memFree=115879624/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=5932008 memory=5385440 CPUtime=161.89 cores=4,6
/proc/30637/stat : 30637 (fscip) S 30634 30637 28195 0 -1 1077936128 767936 0 0 0 15960 229 0 0 20 0 3 0 4590847 6074376192 1346360 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 1483002 1346360 2982 3035 0 1476658 0
[pid=30637/tid=30744] ppid=30634 vsize=5932008 memory=5385440 CPUtime=140.34 cores=4,6
/proc/30637/task/30744/stat : 30744 (fscip) R 30634 30637 28195 0 -1 4194368 368262 0 0 0 13939 95 0 0 20 0 3 0 4592964 6074376192 1346360 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
[pid=30637/tid=30745] ppid=30634 vsize=5932008 memory=14600 CPUtime=0 cores=4,6
/proc/30637/task/30745/stat : 30745 (fscip) S 30634 30637 28195 0 -1 1077936192 37 0 0 0 0 0 0 0 20 0 3 0 4593070 6074376192 1346360 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
Current children cumulated CPU time: 161.89 s
Current children cumulated vsize: 5932008 KiB
Current children cumulated memory: 5385440 KiB

[startup+222.3 s]
/proc/loadavg: 4.10 4.18 4.10 5/247 30781
/proc/meminfo: memFree=115849900/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=5932008 memory=5385440 CPUtime=221.75 cores=4,6
/proc/30637/stat : 30637 (fscip) S 30634 30637 28195 0 -1 1077936128 779661 0 0 0 21945 230 0 0 20 0 3 0 4590847 6074376192 1346360 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 1483002 1346360 2982 3035 0 1476658 0
[pid=30637/tid=30744] ppid=30634 vsize=5932008 memory=5492 CPUtime=200.21 cores=4,6
/proc/30637/task/30744/stat : 30744 (fscip) R 30634 30637 28195 0 -1 4194368 379969 0 0 0 19925 96 0 0 20 0 3 0 4592964 6074376192 1346360 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
[pid=30637/tid=30745] ppid=30634 vsize=5932008 memory=5385440 CPUtime=0 cores=4,6
/proc/30637/task/30745/stat : 30745 (fscip) S 30634 30637 28195 0 -1 1077936192 55 0 0 0 0 0 0 0 20 0 3 0 4593070 6074376192 1346360 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
Current children cumulated CPU time: 221.75 s
Current children cumulated vsize: 5932008 KiB
Current children cumulated memory: 5385440 KiB

[startup+282.3 s]
/proc/loadavg: 4.22 4.19 4.10 5/248 30792
/proc/meminfo: memFree=115826476/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=5932008 memory=5385440 CPUtime=281.62 cores=4,6
/proc/30637/stat : 30637 (fscip) S 30634 30637 28195 0 -1 1077936128 790631 0 0 0 27930 232 0 0 20 0 3 0 4590847 6074376192 1346360 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 1483002 1346360 2982 3035 0 1476658 0
[pid=30637/tid=30744] ppid=30634 vsize=5932008 memory=5385440 CPUtime=260.07 cores=4,6
/proc/30637/task/30744/stat : 30744 (fscip) R 30634 30637 28195 0 -1 4194368 390918 0 0 0 25909 98 0 0 20 0 3 0 4592964 6074376192 1346360 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
[pid=30637/tid=30745] ppid=30634 vsize=5932008 memory=5385440 CPUtime=0 cores=4,6
/proc/30637/task/30745/stat : 30745 (fscip) S 30634 30637 28195 0 -1 1077936192 76 0 0 0 0 0 0 0 20 0 3 0 4593070 6074376192 1346360 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
Current children cumulated CPU time: 281.62 s
Current children cumulated vsize: 5932008 KiB
Current children cumulated memory: 5385440 KiB

[startup+342.3 s]
/proc/loadavg: 4.08 4.15 4.09 5/247 30802

################
# More data... #
################

Current children cumulated memory: 6140292 KiB

[startup+3102.3 s]
/proc/loadavg: 4.15 4.14 4.14 5/249 32021
/proc/meminfo: memFree=95226372/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=6740180 memory=6140636 CPUtime=3093.53 cores=4,6
/proc/30637/stat : 30637 (fscip) S 30634 30637 28195 0 -1 1077936128 2192753 0 0 0 308947 406 0 0 20 0 3 0 4590847 6901944320 1535159 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 1685045 1535159 2982 3035 0 1678701 0
[pid=30637/tid=30744] ppid=30634 vsize=6740180 memory=6140636 CPUtime=3071.92 cores=4,6
/proc/30637/task/30744/stat : 30744 (fscip) R 30634 30637 28195 0 -1 4194368 1792233 0 0 0 306925 267 0 0 20 0 3 0 4592964 6901944320 1535159 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
[pid=30637/tid=30745] ppid=30634 vsize=6740180 memory=14720 CPUtime=0.07 cores=4,6
/proc/30637/task/30745/stat : 30745 (fscip) S 30634 30637 28195 0 -1 1077936192 883 0 0 0 2 5 0 0 20 0 3 0 4593070 6901944320 1535159 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
Current children cumulated CPU time: 3093.53 s
Current children cumulated vsize: 6740180 KiB
Current children cumulated memory: 6140636 KiB

[startup+3162.3 s]
/proc/loadavg: 4.23 4.16 4.15 5/249 32032
/proc/meminfo: memFree=95213416/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=6740180 memory=6140636 CPUtime=3153.36 cores=4,6
/proc/30637/stat : 30637 (fscip) S 30634 30637 28195 0 -1 1077936128 2250970 0 0 0 314925 411 0 0 20 0 3 0 4590847 6901944320 1535159 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 1685045 1535159 2982 3035 0 1678701 0
[pid=30637/tid=30744] ppid=30634 vsize=6740180 memory=11516 CPUtime=3131.75 cores=4,6
/proc/30637/task/30744/stat : 30744 (fscip) R 30634 30637 28195 0 -1 4194368 1850427 0 0 0 312903 272 0 0 20 0 3 0 4592964 6901944320 1535159 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
[pid=30637/tid=30745] ppid=30634 vsize=6740180 memory=6140636 CPUtime=0.07 cores=4,6
/proc/30637/task/30745/stat : 30745 (fscip) S 30634 30637 28195 0 -1 1077936192 906 0 0 0 2 5 0 0 20 0 3 0 4593070 6901944320 1535159 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
Current children cumulated CPU time: 3153.36 s
Current children cumulated vsize: 6740180 KiB
Current children cumulated memory: 6140636 KiB

[startup+3222.3 s]
/proc/loadavg: 4.14 4.15 4.14 5/249 32042
/proc/meminfo: memFree=95198672/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=6744368 memory=6150816 CPUtime=3213.21 cores=4,6
/proc/30637/stat : 30637 (fscip) S 30634 30637 28195 0 -1 1077936128 2315985 0 0 0 320904 417 0 0 20 0 3 0 4590847 6906232832 1537704 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 1686092 1537704 2982 3035 0 1679748 0
[pid=30637/tid=30744] ppid=30634 vsize=6744368 memory=14720 CPUtime=3191.59 cores=4,6
/proc/30637/task/30744/stat : 30744 (fscip) R 30634 30637 28195 0 -1 4194368 1915426 0 0 0 318881 278 0 0 20 0 3 0 4592964 6906232832 1537704 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
[pid=30637/tid=30745] ppid=30634 vsize=6744368 memory=6141244 CPUtime=0.07 cores=4,6
/proc/30637/task/30745/stat : 30745 (fscip) S 30634 30637 28195 0 -1 1077936192 922 0 0 0 2 5 0 0 20 0 3 0 4593070 6906232832 1537704 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
Current children cumulated CPU time: 3213.21 s
Current children cumulated vsize: 6744368 KiB
Current children cumulated memory: 6150816 KiB

[startup+3282.3 s]
/proc/loadavg: 4.09 4.13 4.14 5/249 32052
/proc/meminfo: memFree=95195336/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=6744684 memory=6151132 CPUtime=3273.05 cores=4,6
/proc/30637/stat : 30637 (fscip) S 30634 30637 28195 0 -1 1077936128 2361370 0 0 0 326883 422 0 0 20 0 3 0 4590847 6906556416 1537783 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 1686171 1537783 2982 3035 0 1679827 0
[pid=30637/tid=30744] ppid=30634 vsize=6744684 memory=6151132 CPUtime=3251.43 cores=4,6
/proc/30637/task/30744/stat : 30744 (fscip) R 30634 30637 28195 0 -1 4194368 1960788 0 0 0 324861 282 0 0 20 0 3 0 4592964 6906556416 1537783 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
[pid=30637/tid=30745] ppid=30634 vsize=6744684 memory=6151132 CPUtime=0.07 cores=4,6
/proc/30637/task/30745/stat : 30745 (fscip) S 30634 30637 28195 0 -1 1077936192 945 0 0 0 2 5 0 0 20 0 3 0 4593070 6906556416 1537783 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
Current children cumulated CPU time: 3273.05 s
Current children cumulated vsize: 6744684 KiB
Current children cumulated memory: 6151132 KiB

[startup+3342.3 s]
/proc/loadavg: 4.14 4.14 4.14 5/249 32447
/proc/meminfo: memFree=95180104/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=6744684 memory=6163568 CPUtime=3332.89 cores=4,6
/proc/30637/stat : 30637 (fscip) S 30634 30637 28195 0 -1 1077936128 2430978 0 0 0 332863 426 0 0 20 0 3 0 4590847 6906556416 1540892 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 1686171 1540892 2982 3035 0 1679827 0
[pid=30637/tid=30744] ppid=30634 vsize=6744684 memory=6163568 CPUtime=3311.27 cores=4,6
/proc/30637/task/30744/stat : 30744 (fscip) R 30634 30637 28195 0 -1 4194368 2030380 0 0 0 330840 287 0 0 20 0 3 0 4592964 6906556416 1540892 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
[pid=30637/tid=30745] ppid=30634 vsize=6744684 memory=6163568 CPUtime=0.07 cores=4,6
/proc/30637/task/30745/stat : 30745 (fscip) S 30634 30637 28195 0 -1 1077936192 961 0 0 0 2 5 0 0 20 0 3 0 4593070 6906556416 1540892 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
Current children cumulated CPU time: 3332.89 s
Current children cumulated vsize: 6744684 KiB
Current children cumulated memory: 6163568 KiB

[startup+3402.3 s]
/proc/loadavg: 4.10 4.13 4.13 5/248 32459
/proc/meminfo: memFree=95168940/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=6747200 memory=6172032 CPUtime=3392.73 cores=4,6
/proc/30637/stat : 30637 (fscip) S 30634 30637 28195 0 -1 1077936128 2492413 0 0 0 338843 430 0 0 20 0 3 0 4590847 6909132800 1543008 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 1686800 1543008 2982 3035 0 1680456 0
[pid=30637/tid=30744] ppid=30634 vsize=6747200 memory=6163568 CPUtime=3371.11 cores=4,6
/proc/30637/task/30744/stat : 30744 (fscip) R 30634 30637 28195 0 -1 4194368 2091797 0 0 0 336820 291 0 0 20 0 3 0 4592964 6909132800 1543008 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
[pid=30637/tid=30745] ppid=30634 vsize=6747200 memory=6166512 CPUtime=0.07 cores=4,6
/proc/30637/task/30745/stat : 30745 (fscip) S 30634 30637 28195 0 -1 1077936192 979 0 0 0 2 5 0 0 20 0 3 0 4593070 6909132800 1543008 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
Current children cumulated CPU time: 3392.73 s
Current children cumulated vsize: 6747200 KiB
Current children cumulated memory: 6172032 KiB

[startup+3462.3 s]
/proc/loadavg: 4.15 4.13 4.13 5/248 32471
/proc/meminfo: memFree=95043496/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=6747200 memory=6172124 CPUtime=3452.57 cores=4,6
/proc/30637/stat : 30637 (fscip) S 30634 30637 28195 0 -1 1077936128 2552053 0 0 0 344823 434 0 0 20 0 3 0 4590847 6909132800 1543031 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 1686800 1543031 2982 3035 0 1680456 0
[pid=30637/tid=30744] ppid=30634 vsize=6747200 memory=6163568 CPUtime=3430.95 cores=4,6
/proc/30637/task/30744/stat : 30744 (fscip) R 30634 30637 28195 0 -1 4194368 2151416 0 0 0 342801 294 0 0 20 0 3 0 4592964 6909132800 1543031 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
[pid=30637/tid=30745] ppid=30634 vsize=6747200 memory=14720 CPUtime=0.07 cores=4,6
/proc/30637/task/30745/stat : 30745 (fscip) S 30634 30637 28195 0 -1 1077936192 1000 0 0 0 2 5 0 0 20 0 3 0 4593070 6909132800 1543031 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
Current children cumulated CPU time: 3452.57 s
Current children cumulated vsize: 6747200 KiB
Current children cumulated memory: 6172124 KiB

[startup+3522.3 s]
/proc/loadavg: 4.12 4.12 4.13 5/251 32652
/proc/meminfo: memFree=95031024/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=6747200 memory=6172376 CPUtime=3512.41 cores=4,6
/proc/30637/stat : 30637 (fscip) S 30634 30637 28195 0 -1 1077936128 2610772 0 0 0 350803 438 0 0 20 0 3 0 4590847 6909132800 1543094 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 1686800 1543094 2982 3035 0 1680456 0
[pid=30637/tid=30744] ppid=30634 vsize=6747200 memory=6172124 CPUtime=3490.78 cores=4,6
/proc/30637/task/30744/stat : 30744 (fscip) R 30634 30637 28195 0 -1 4194368 2210111 0 0 0 348780 298 0 0 20 0 3 0 4592964 6909132800 1543094 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
[pid=30637/tid=30745] ppid=30634 vsize=6747200 memory=11516 CPUtime=0.08 cores=4,6
/proc/30637/task/30745/stat : 30745 (fscip) S 30634 30637 28195 0 -1 1077936192 1024 0 0 0 2 6 0 0 20 0 3 0 4593070 6909132800 1543094 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
Current children cumulated CPU time: 3512.41 s
Current children cumulated vsize: 6747200 KiB
Current children cumulated memory: 6172376 KiB

[startup+3582.3 s]
/proc/loadavg: 4.15 4.13 4.13 5/251 32663
/proc/meminfo: memFree=95018960/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=6747972 memory=6173148 CPUtime=3572.24 cores=4,6
/proc/30637/stat : 30637 (fscip) S 30634 30637 28195 0 -1 1077936128 2681494 0 0 0 356780 444 0 0 20 0 3 0 4590847 6909923328 1543287 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 1686993 1543287 2982 3035 0 1680649 0
[pid=30637/tid=30744] ppid=30634 vsize=6747972 memory=12624 CPUtime=3550.62 cores=4,6
/proc/30637/task/30744/stat : 30744 (fscip) R 30634 30637 28195 0 -1 4194368 2280817 0 0 0 354758 304 0 0 20 0 3 0 4592964 6909923328 1543287 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
[pid=30637/tid=30745] ppid=30634 vsize=6747972 memory=11516 CPUtime=0.08 cores=4,6
/proc/30637/task/30745/stat : 30745 (fscip) S 30634 30637 28195 0 -1 1077936192 1040 0 0 0 2 6 0 0 20 0 3 0 4593070 6909923328 1543287 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
Current children cumulated CPU time: 3572.24 s
Current children cumulated vsize: 6747972 KiB
Current children cumulated memory: 6173148 KiB



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+3610.2 s]
/proc/loadavg: 4.10 4.12 4.12 5/252 32772
/proc/meminfo: memFree=103097156/131655788 swapFree=131071216/131071996
[pid=30637] ppid=30634 vsize=6747972 memory=6173148 CPUtime=3600.07 cores=4,6
/proc/30637/stat : 30637 (fscip) S 30634 30637 28195 0 -1 1077936128 2707693 0 0 0 359562 445 0 0 20 0 3 0 4590847 6909923328 1543287 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 17 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
/proc/30637/statm: 1686993 1543287 2982 3035 0 1680649 0
[pid=30637/tid=30744] ppid=30634 vsize=6747972 memory=-5155631301594475397 CPUtime=3578.45 cores=4,6
/proc/30637/task/30744/stat : 30744 (fscip) R 30634 30637 28195 0 -1 4194368 2307007 0 0 0 357540 305 0 0 20 0 3 0 4592964 6909923328 1543287 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 0 0 0 -1 4 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
[pid=30637/tid=30745] ppid=30634 vsize=6747972 memory=6173148 CPUtime=0.08 cores=4,6
/proc/30637/task/30745/stat : 30745 (fscip) S 30634 30637 28195 0 -1 1077936192 1049 0 0 0 2 6 0 0 20 0 3 0 4593070 6909923328 1543287 134679101440 4194304 16623229 140732811024208 0 0 0 0 0 2 1 0 0 -1 6 0 0 0 0 0 18838560 18947056 45350912 140732811027482 140732811027598 140732811027598 140732811034608 0
Current children cumulated CPU time: 3600.07 s
Current children cumulated vsize: 6747972 KiB
Current children cumulated memory: 6173148 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

[startup+3610.27 s]
# the end of solver process 30637 was just reported to runsolver
# Child ended because it received signal 15 (SIGTERM)

# cumulated CPU time of all completed processes:  user=3595.63 s, system=4.51873 s

Solver just ended.
??? end of watcher thread
Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

Real time (s): 3610.27
CPU time (s): 3600.15
CPU user time (s): 3595.63
CPU system time (s): 4.51873
CPU usage (%): 99.7197
Max. virtual memory (cumulated for all children) (KiB): 6875612
Max. memory (cumulated for all children) (KiB): 6173148

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 3595.63
system time used= 4.51873
maximum resident set size= 6231888
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2707694
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3657
involuntary context switches= 1510


# summary of solver processes directly reported to runsolver:
#   pid: 30637
#   total CPU time (s): 3600.15
#   total CPU user time (s): 3595.63
#   total CPU system time (s): 4.51873

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 19.2282 second user time and 37.3129 second system time

The end

Launcher Data 

Begin job on nodeC007 at 2024-07-31 01:30:02
IDJOB=4464184
IDBENCH=155402
IDSOLVER=3092
FILE ID=nodeC007/4464184-1722382202
RUNJOBID= nodeC007-1722378594-28206
SLURM_JOB_ID= 8249837
Free space on /tmp= 723656 MiB

SOLVER NAME= FiberSCIP 2024-07-30
BENCH NAME= PB24/normalized-PB11/OPT-LIN/kullmann/normalized-mul_8_14_shg.opb
COMMAND LINE= ./fscip settings/racing.prm BENCHNAME -qsol -t TIMEOUT -sth 1
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp --file-size-limit 102400 -w /tmp/evaluation-result-4464184-1722382202/watcher-4464184-1722382202 -o /tmp/evaluation-result-4464184-1722382202/solver-4464184-1722382202 -C 3600 -W 3700 -M 31000 --bin-var /tmp/runsolver-exch-nodeC007-1722378594-28206 --watchdog 3760  ./fscip settings/racing.prm HOME/instance-4464184-1722382202.opb -qsol -t 3600 -sth 1

TIME LIMIT= 3600 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 2

MD5SUM BENCH= c0e25a71d8726d42148f3e780c01ff44
RANDOM SEED=1367756045

nodeC007.alineos.net Linux 4.18.0-301.1.el8.x86_64 #1 SMP Tue Apr 13 16:24:22 UTC 2021

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00001f
cpu MHz		: 3664.934
cache size	: 15360 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 6983.58
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00001f
cpu MHz		: 3666.644
cache size	: 15360 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 6983.35
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00001f
cpu MHz		: 1197.295
cache size	: 15360 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 6983.58
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00001f
cpu MHz		: 1197.272
cache size	: 15360 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 6983.35
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00001f
cpu MHz		: 1412.195
cache size	: 15360 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 6983.58
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00001f
cpu MHz		: 3669.804
cache size	: 15360 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 6983.35
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00001f
cpu MHz		: 1466.270
cache size	: 15360 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 6983.58
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00001f
cpu MHz		: 1197.221
cache size	: 15360 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 6983.35
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       131655788 kB
MemFree:        129175060 kB
MemAvailable:   129453548 kB
Buffers:           84716 kB
Cached:          1033260 kB
SwapCached:          100 kB
Active:           245368 kB
Inactive:        1812444 kB
Active(anon):       1492 kB
Inactive(anon):   959456 kB
Active(file):     243876 kB
Inactive(file):   852988 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:      131071996 kB
SwapFree:       131071216 kB
Dirty:            130992 kB
Writeback:             0 kB
AnonPages:        939768 kB
Mapped:           125136 kB
Shmem:             21076 kB
KReclaimable:      43648 kB
Slab:             119512 kB
SReclaimable:      43648 kB
SUnreclaim:        75864 kB
KernelStack:        4000 kB
PageTables:        11072 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    196899888 kB
Committed_AS:     697560 kB
VmallocTotal:   34359738367 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
Percpu:             6272 kB
HardwareCorrupted:     0 kB
AnonHugePages:    151552 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
FileHugePages:         0 kB
FilePmdMapped:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB
DirectMap4k:      174920 kB
DirectMap2M:     7069696 kB
DirectMap1G:    128974848 kB

Free space on /tmp at the end= 723504 MiB
End job on nodeC007 at 2024-07-31 02:30:13