Trace number 4464181

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
FiberSCIP 2024-07-30? 3587.33 3599.27

General information on the benchmark

NamePB24/normalized-PB11/OPT-LIN/
heinz/normalized-stp3d.opb
MD5SUMa4b0cd988cdc47ea50384dbc6c682a43
Bench CategoryOPT-LIN (optimization problem, linear constraints, no OPT/UNSAT certificate)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Has Objective FunctionYES
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables204880
Total number of constraints159488
Number of constraints which are clauses129232
Number of constraints which are cardinality constraints (but not clauses)5805
Number of constraints which are nor clauses,nor cardinality constraints24451
Minimum length of a constraint2
Maximum length of a constraint120
Number of terms in the objective function 75648
Biggest coefficient in the objective function 100303
Number of bits for the biggest coefficient in the objective function 17
Sum of the numbers in the objective function 7576266096
Number of bits of the sum of numbers in the objective function 33
Biggest number in a constraint 100303
Number of bits of the biggest number in a constraint 17
Biggest sum of numbers in a constraint 7576266096
Number of bits of the biggest sum of numbers33
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c The following solver is parallelized by UG version 1.0.0 [GitHash: e6e2b6ff-dirty]
0.00/0.00	c SCIP version 10.0.0 [precision: 8 byte] [memory: block] [mode: optimized] [LP solver: Soplex 7.1.1] [GitHash: 020d91813d]
0.00/0.00	c Copyright (c) 2002-2024 Zuse Institute Berlin (ZIB)
0.00/0.00	c External libraries: 
0.00/0.00	c   Soplex 7.1.1         Linear Programming Solver developed at Zuse Institute Berlin (soplex.zib.de) [GitHash: b4905e82]
0.00/0.00	c   CppAD 20180000.0     Algorithmic Differentiation of C++ algorithms developed by B. Bell (github.com/coin-or/CppAD)
0.00/0.00	c   AMPL/MP 690e9e7      AMPL .nl file reader library (github.com/ampl/mp)
0.00/0.00	c   PaPILO 2.4.0         parallel presolve for integer and linear optimization (github.com/scipopt/papilo)
0.00/0.00	c   Nauty 2.8.8          Computing Graph Automorphism Groups by Brendan D. McKay (users.cecs.anu.edu.au/~bdm/nauty)
0.00/0.00	c   sassy 1.1            Symmetry preprocessor by Markus Anders (github.com/markusa4/sassy)
1.09/1.14	c Default LC presolving (default).
1.09/1.14	c ** Before presolving: virtualMemUsedAtLc = 960696777, getVmSize() = 672477184, SCIPgetMemUsed() = 623544604, SCIPgetMemTotal() = 630974068, SCIPgetMemExternEstim() = 329722709
1.09/1.14	c ** set memory limit for presolving in LC to 8.7958e+12 for SCIP **
5.27/5.37	c ** Estimated virtualMemUsedAtSolver = 839246049, getVmSize() = 1020784640, SCIPgetMemUsed() = 724716301, SCIPgetMemTotal() = 929146365, SCIPgetMemExternEstim() = 329722709
5.27/5.37	c ** set memory limit for solvers to 4.39802e+12 for each SCIP **
5.27/5.37	c Original Problem   :
5.27/5.37	c   Problem name     : instance-4464181-1722382201
5.27/5.37	c   Variables        : 204880 (204880 binary, 0 integer, 0 implicit integer, 0 continuous)
5.27/5.37	c   Constraints      : 159488
5.27/5.37	c   Objective sense  : minimize
5.27/5.37	c Presolved Problem  :
5.27/5.37	c   Variables        : 136916 (136916 binary, 0 integer, 0 implicit integer, 0 continuous)
5.27/5.37	c   Constraints      : 97472
5.27/5.37	c Constraints        : Number
5.27/5.37	c   knapsack         : 1 
5.27/5.37	c   setppc           : 73321 
5.27/5.37	c   linear           : 24150 
5.27/5.37	c ** Instance transfer method used: 0
5.67/5.75	c ** ParaScipInstance copy does not increase the number of variables. **
5.97/6.06	c LC is working with racing ramp-up and with rebuilding tree after racing.
5.97/6.06	c                                Nodes    Active                                            
5.97/6.06	c      Time          Nodes        Left   Solvers     Best Integer        Best Node        Gap     Best Node(S)     Gap(S)
3586.06/3598.07	c      3598              0           1         0                --179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000          -
3586.76/3598.75	c SCIP Status        : c solving was interrupted [hard time limit reached]
3586.76/3598.75	c Total Time         : 3598.7600
3586.76/3598.75	c   solving          : 3598.7600
3586.76/3598.75	c   presolving       : 4.2320 (included in solving)
3586.76/3598.75	c B&B Tree           :
3586.76/3598.75	c   nodes (total)    : 1
3586.76/3598.75	c Solution           :
3586.76/3598.75	c   Solutions found  : 0
3586.76/3598.75	c   Primal Bound     :          -
3586.76/3598.75	c   Dual Bound       :          -
3586.76/3598.75	c Gap                :          -
3587.25/3599.26	s UNKNOWN

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver version 3.4.2 (svn: 4680) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 63968 MiB (61191 MiB free)
  memory of node 1: 64473 MiB (62953 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp --file-size-limit 102400 -w /tmp/evaluation-result-4464181-1722382201/watcher-4464181-1722382201 -o /tmp/evaluation-result-4464181-1722382201/solver-4464181-1722382201 -C 3600 -W 3700 -M 31000 --bin-var /tmp/runsolver-exch-nodeC034-1722378597-30625 --watchdog 3760 ./fscip settings/racing.prm HOME/instance-4464181-1722382201.opb -qsol -t 3600 -sth 1 

running on 2 cores: 1,3

CPU governors:
  performance: 1 3

Enforcing FSIZE limit: 104857600 KiB
Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 3600 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 3630 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 3700 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Linux 4.18.0-301.1.el8.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=34369, runsolver pid=34366

[startup+0.100996 s]*
/proc/loadavg: 4.12 4.13 4.04 5/263 34370
/proc/meminfo: memFree=127030716/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=117248 memory=102204 CPUtime=0.09 cores=1,3
/proc/34369/stat : 34369 (fscip) R 34366 34369 30613 0 -1 4194304 23421 0 0 0 6 3 0 0 20 0 1 0 4824414 120061952 25551 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 29312 25551 2347 3035 0 25708 0
Current children cumulated CPU time: 0.09 s
Current children cumulated vsize: 117248 KiB
Current children cumulated memory: 102204 KiB

[startup+0.205909 s]*
/proc/loadavg: 4.12 4.13 4.04 5/263 34370
/proc/meminfo: memFree=126982828/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=167756 memory=152712 CPUtime=0.2 cores=1,3
/proc/34369/stat : 34369 (fscip) R 34366 34369 30613 0 -1 4194304 36080 0 0 0 15 5 0 0 20 0 1 0 4824414 171782144 38178 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 41939 38178 2347 3035 0 38335 0
Current children cumulated CPU time: 0.2 s
Current children cumulated vsize: 167756 KiB
Current children cumulated memory: 152712 KiB

[startup+0.306001 s]*
/proc/loadavg: 4.12 4.13 4.04 5/263 34370
/proc/meminfo: memFree=126928264/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=223328 memory=207428 CPUtime=0.29 cores=1,3
/proc/34369/stat : 34369 (fscip) R 34366 34369 30613 0 -1 4194304 49730 0 0 0 22 7 0 0 20 0 1 0 4824414 228687872 51857 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 55832 51857 2347 3035 0 52228 0
Current children cumulated CPU time: 0.29 s
Current children cumulated vsize: 223328 KiB
Current children cumulated memory: 207428 KiB

[startup+0.700197 s]
/proc/loadavg: 4.12 4.13 4.04 5/263 34370
/proc/meminfo: memFree=126837084/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=445800 memory=434016 CPUtime=0.68 cores=1,3
/proc/34369/stat : 34369 (fscip) R 34366 34369 30613 0 -1 4194304 110082 0 0 0 53 15 0 0 20 0 1 0 4824414 456499200 108504 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 111450 108504 2555 3035 0 107846 0
Current children cumulated CPU time: 0.68 s
Current children cumulated vsize: 445800 KiB
Current children cumulated memory: 434016 KiB

[startup+1.50019 s]
/proc/loadavg: 4.11 4.13 4.04 5/263 34372
/proc/meminfo: memFree=126537828/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=711132 memory=690224 CPUtime=1.48 cores=1,3
/proc/34369/stat : 34369 (fscip) R 34366 34369 30613 0 -1 4194304 174111 0 0 0 124 24 0 0 20 0 1 0 4824414 728199168 172556 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 177783 172556 2614 3035 0 174179 0
Current children cumulated CPU time: 1.48 s
Current children cumulated vsize: 711132 KiB
Current children cumulated memory: 690224 KiB

[startup+3.10614 s]
/proc/loadavg: 4.11 4.13 4.04 5/263 34372
/proc/meminfo: memFree=126383164/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=788996 memory=764068 CPUtime=3.08 cores=1,3
/proc/34369/stat : 34369 (fscip) R 34366 34369 30613 0 -1 4194304 197884 0 0 0 278 30 0 0 20 0 1 0 4824414 807931904 191017 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 197249 191017 2726 3035 0 193645 0
Current children cumulated CPU time: 3.08 s
Current children cumulated vsize: 788996 KiB
Current children cumulated memory: 764068 KiB

[startup+6.30099 s]
/proc/loadavg: 4.10 4.12 4.04 5/264 34373
/proc/meminfo: memFree=126045664/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=1404328 memory=1277896 CPUtime=6.26 cores=1,3
/proc/34369/stat : 34369 (fscip) S 34366 34369 30613 0 -1 1077936128 358034 0 0 0 573 53 0 0 20 0 3 0 4824414 1438031872 319474 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 351082 319474 2822 3035 0 342438 0
[pid=34369/tid=34373] ppid=34366 vsize=1404328 memory=6081941222239856994 CPUtime=0.54 cores=1,3
/proc/34369/task/34373/stat : 34373 (fscip) R 34366 34369 30613 0 -1 4194368 92326 0 0 0 42 12 0 0 20 0 3 0 4824952 1438031872 319474 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 -1 3 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
Current children cumulated CPU time: 6.26 s
Current children cumulated vsize: 1404328 KiB
Current children cumulated memory: 1277896 KiB

[startup+12.701 s]
/proc/loadavg: 4.18 4.14 4.05 5/265 34375
/proc/meminfo: memFree=125592276/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=1704068 memory=1539880 CPUtime=12.61 cores=1,3
/proc/34369/stat : 34369 (fscip) S 34366 34369 30613 0 -1 1077936128 465124 0 0 0 1195 66 0 0 20 0 3 0 4824414 1744965632 384970 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 426017 384970 2822 3035 0 412954 0
[pid=34369/tid=34373] ppid=34366 vsize=1704068 memory=11408 CPUtime=6.87 cores=1,3
/proc/34369/task/34373/stat : 34373 (fscip) R 34366 34369 30613 0 -1 4194368 199411 0 0 0 663 24 0 0 20 0 3 0 4824952 1744965632 384970 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 -1 3 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
[pid=34369/tid=34374] ppid=34366 vsize=1704068 memory=0 CPUtime=0 cores=1,3
/proc/34369/task/34374/stat : 34374 (fscip) S 34366 34369 30613 0 -1 1077936192 5 0 0 0 0 0 0 0 20 0 3 0 4825021 1744965632 384970 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 1 0 0 -1 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
Current children cumulated CPU time: 12.61 s
Current children cumulated vsize: 1704068 KiB
Current children cumulated memory: 1539880 KiB

[startup+25.501 s]
/proc/loadavg: 4.15 4.13 4.04 5/265 34377
/proc/meminfo: memFree=125568120/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=1704068 memory=1563340 CPUtime=25.28 cores=1,3
/proc/34369/stat : 34369 (fscip) S 34366 34369 30613 0 -1 1077936128 552089 0 0 0 2456 72 0 0 20 0 3 0 4824414 1744965632 390835 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 426017 390835 2822 3035 0 418836 0
[pid=34369/tid=34373] ppid=34366 vsize=1704068 memory=11408 CPUtime=19.54 cores=1,3
/proc/34369/task/34373/stat : 34373 (fscip) R 34366 34369 30613 0 -1 4194368 286366 0 0 0 1924 30 0 0 20 0 3 0 4824952 1744965632 390835 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 -1 3 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
[pid=34369/tid=34374] ppid=34366 vsize=1704068 memory=1560472 CPUtime=0 cores=1,3
/proc/34369/task/34374/stat : 34374 (fscip) S 34366 34369 30613 0 -1 1077936192 15 0 0 0 0 0 0 0 20 0 3 0 4825021 1744965632 390835 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 1 0 0 -1 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
Current children cumulated CPU time: 25.28 s
Current children cumulated vsize: 1704068 KiB
Current children cumulated memory: 1563340 KiB

[startup+51.106 s]
/proc/loadavg: 4.09 4.12 4.04 5/264 34383
/proc/meminfo: memFree=125194080/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=2031748 memory=1877644 CPUtime=50.62 cores=1,3
/proc/34369/stat : 34369 (fscip) S 34366 34369 30613 0 -1 1077936128 769160 0 0 0 4970 92 0 0 20 0 3 0 4824414 2080509952 469411 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 507937 469411 2950 3035 0 498413 0
[pid=34369/tid=34373] ppid=34366 vsize=2031748 memory=1876768 CPUtime=44.89 cores=1,3
/proc/34369/task/34373/stat : 34373 (fscip) R 34366 34369 30613 0 -1 4194368 503425 0 0 0 4439 50 0 0 20 0 3 0 4824952 2080509952 469411 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 -1 3 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
[pid=34369/tid=34374] ppid=34366 vsize=2031748 memory=1865424 CPUtime=0 cores=1,3
/proc/34369/task/34374/stat : 34374 (fscip) S 34366 34369 30613 0 -1 1077936192 27 0 0 0 0 0 0 0 20 0 3 0 4825021 2080509952 469411 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 1 0 0 -1 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
Current children cumulated CPU time: 50.62 s
Current children cumulated vsize: 2031748 KiB
Current children cumulated memory: 1877644 KiB

[startup+102.301 s]
/proc/loadavg: 4.16 4.13 4.05 5/264 34391
/proc/meminfo: memFree=125191908/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=2031748 memory=1877664 CPUtime=101.31 cores=1,3
/proc/34369/stat : 34369 (fscip) S 34366 34369 30613 0 -1 1077936128 825974 0 0 0 10026 105 0 0 20 0 3 0 4824414 2080509952 469416 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 507937 469416 2950 3035 0 498413 0
[pid=34369/tid=34373] ppid=34366 vsize=2031748 memory=1877660 CPUtime=95.57 cores=1,3
/proc/34369/task/34373/stat : 34373 (fscip) R 34366 34369 30613 0 -1 4194368 560222 0 0 0 9494 63 0 0 20 0 3 0 4824952 2080509952 469416 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 -1 3 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
[pid=34369/tid=34374] ppid=34366 vsize=2031748 memory=11408 CPUtime=0 cores=1,3
/proc/34369/task/34374/stat : 34374 (fscip) S 34366 34369 30613 0 -1 1077936192 44 0 0 0 0 0 0 0 20 0 3 0 4825021 2080509952 469416 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 1 0 0 -1 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
Current children cumulated CPU time: 101.31 s
Current children cumulated vsize: 2031748 KiB
Current children cumulated memory: 1877664 KiB

[startup+162.3 s]
/proc/loadavg: 4.12 4.12 4.05 5/264 34449
/proc/meminfo: memFree=124178820/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=2097284 memory=1881012 CPUtime=161.03 cores=1,3
/proc/34369/stat : 34369 (fscip) S 34366 34369 30613 0 -1 1077936128 886463 0 0 0 15983 120 0 0 20 0 3 0 4824414 2147618816 470253 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 524321 470253 2950 3035 0 499231 0
[pid=34369/tid=34373] ppid=34366 vsize=2097284 memory=0 CPUtime=155.29 cores=1,3
/proc/34369/task/34373/stat : 34373 (fscip) R 34366 34369 30613 0 -1 4194368 620698 0 0 0 15451 78 0 0 20 0 3 0 4824952 2147618816 470253 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 -1 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
[pid=34369/tid=34374] ppid=34366 vsize=2097284 memory=1880960 CPUtime=0 cores=1,3
/proc/34369/task/34374/stat : 34374 (fscip) S 34366 34369 30613 0 -1 1077936192 57 0 0 0 0 0 0 0 20 0 3 0 4825021 2147618816 470253 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 1 0 0 -1 3 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
Current children cumulated CPU time: 161.03 s
Current children cumulated vsize: 2097284 KiB
Current children cumulated memory: 1881012 KiB

[startup+222.3 s]
/proc/loadavg: 4.08 4.11 4.05 5/262 34459
/proc/meminfo: memFree=123631532/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=2097284 memory=1881476 CPUtime=220.86 cores=1,3
/proc/34369/stat : 34369 (fscip) S 34366 34369 30613 0 -1 1077936128 953517 0 0 0 21953 133 0 0 20 0 3 0 4824414 2147618816 470369 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 524321 470369 2950 3035 0 499345 0
[pid=34369/tid=34373] ppid=34366 vsize=2097284 memory=1877876 CPUtime=215.13 cores=1,3
/proc/34369/task/34373/stat : 34373 (fscip) R 34366 34369 30613 0 -1 4194368 687740 0 0 0 21422 91 0 0 20 0 3 0 4824952 2147618816 470369 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 -1 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
[pid=34369/tid=34374] ppid=34366 vsize=2097284 memory=1877876 CPUtime=0 cores=1,3
/proc/34369/task/34374/stat : 34374 (fscip) S 34366 34369 30613 0 -1 1077936192 69 0 0 0 0 0 0 0 20 0 3 0 4825021 2147618816 470369 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 1 0 0 -1 3 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
Current children cumulated CPU time: 220.86 s
Current children cumulated vsize: 2097284 KiB
Current children cumulated memory: 1881476 KiB

[startup+282.3 s]
/proc/loadavg: 4.20 4.13 4.06 5/264 34473
/proc/meminfo: memFree=121769580/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=2097284 memory=1891244 CPUtime=280.7 cores=1,3
/proc/34369/stat : 34369 (fscip) S 34366 34369 30613 0 -1 1077936128 998127 0 0 0 27930 140 0 0 20 0 3 0 4824414 2147618816 472811 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 524321 472811 2950 3035 0 502941 0
[pid=34369/tid=34373] ppid=34366 vsize=2097284 memory=12860 CPUtime=274.97 cores=1,3
/proc/34369/task/34373/stat : 34373 (fscip) R 34366 34369 30613 0 -1 4194368 732338 0 0 0 27399 98 0 0 20 0 3 0 4824952 2147618816 472811 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 -1 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
[pid=34369/tid=34374] ppid=34366 vsize=2097284 memory=1881364 CPUtime=0 cores=1,3
/proc/34369/task/34374/stat : 34374 (fscip) S 34366 34369 30613 0 -1 1077936192 81 0 0 0 0 0 0 0 20 0 3 0 4825021 2147618816 472811 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 1 0 0 -1 3 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0

################
# More data... #
################

/proc/loadavg: 4.12 4.17 4.12 5/266 36080
/proc/meminfo: memFree=109605936/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=2386436 memory=2199936 CPUtime=3390.93 cores=1,3
/proc/34369/stat : 34369 (fscip) S 34366 34369 30613 0 -1 1077936128 2947625 0 0 0 338816 277 0 0 20 0 3 0 4824414 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 596609 549984 2982 3035 0 585858 0
[pid=34369/tid=34373] ppid=34366 vsize=2386436 memory=12860 CPUtime=3385.15 cores=1,3
/proc/34369/task/34373/stat : 34373 (fscip) R 34366 34369 30613 0 -1 4194368 2681368 0 0 0 338283 232 0 0 20 0 3 0 4824952 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 -1 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
[pid=34369/tid=34374] ppid=34366 vsize=2386436 memory=2199936 CPUtime=0.04 cores=1,3
/proc/34369/task/34374/stat : 34374 (fscip) S 34366 34369 30613 0 -1 1077936192 549 0 0 0 1 3 0 0 20 0 3 0 4825021 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 1 0 0 -1 3 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
Current children cumulated CPU time: 3390.93 s
Current children cumulated vsize: 2386436 KiB
Current children cumulated memory: 2199936 KiB

[startup+3462.3 s]
/proc/loadavg: 4.15 4.17 4.12 5/266 36091
/proc/meminfo: memFree=109605040/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=2386436 memory=2199936 CPUtime=3450.76 cores=1,3
/proc/34369/stat : 34369 (fscip) S 34366 34369 30613 0 -1 1077936128 2949907 0 0 0 344799 277 0 0 20 0 3 0 4824414 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 596609 549984 2982 3035 0 585858 0
[pid=34369/tid=34373] ppid=34366 vsize=2386436 memory=12860 CPUtime=3444.98 cores=1,3
/proc/34369/task/34373/stat : 34373 (fscip) R 34366 34369 30613 0 -1 4194368 2683642 0 0 0 344266 232 0 0 20 0 3 0 4824952 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 -1 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
[pid=34369/tid=34374] ppid=34366 vsize=2386436 memory=2199936 CPUtime=0.04 cores=1,3
/proc/34369/task/34374/stat : 34374 (fscip) S 34366 34369 30613 0 -1 1077936192 557 0 0 0 1 3 0 0 20 0 3 0 4825021 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 1 0 0 -1 3 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
Current children cumulated CPU time: 3450.76 s
Current children cumulated vsize: 2386436 KiB
Current children cumulated memory: 2199936 KiB

[startup+3522.3 s]
/proc/loadavg: 4.11 4.15 4.11 5/267 36271
/proc/meminfo: memFree=109603444/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=2386436 memory=2199936 CPUtime=3510.58 cores=1,3
/proc/34369/stat : 34369 (fscip) S 34366 34369 30613 0 -1 1077936128 2953309 0 0 0 350780 278 0 0 20 0 3 0 4824414 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 596609 549984 2982 3035 0 585858 0
[pid=34369/tid=34373] ppid=34366 vsize=2386436 memory=11408 CPUtime=3504.79 cores=1,3
/proc/34369/task/34373/stat : 34373 (fscip) R 34366 34369 30613 0 -1 4194368 2687034 0 0 0 350247 232 0 0 20 0 3 0 4824952 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 -1 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
[pid=34369/tid=34374] ppid=34366 vsize=2386436 memory=2199936 CPUtime=0.04 cores=1,3
/proc/34369/task/34374/stat : 34374 (fscip) S 34366 34369 30613 0 -1 1077936192 567 0 0 0 1 3 0 0 20 0 3 0 4825021 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 1 0 0 -1 3 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
Current children cumulated CPU time: 3510.58 s
Current children cumulated vsize: 2386436 KiB
Current children cumulated memory: 2199936 KiB

[startup+3582.3 s]
/proc/loadavg: 4.08 4.14 4.11 5/267 36282
/proc/meminfo: memFree=109600664/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=2386436 memory=2199936 CPUtime=3570.41 cores=1,3
/proc/34369/stat : 34369 (fscip) S 34366 34369 30613 0 -1 1077936128 2955905 0 0 0 356763 278 0 0 20 0 3 0 4824414 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 596609 549984 2982 3035 0 585858 0
[pid=34369/tid=34373] ppid=34366 vsize=2386436 memory=2199936 CPUtime=3564.63 cores=1,3
/proc/34369/task/34373/stat : 34373 (fscip) R 34366 34369 30613 0 -1 4194368 2689622 0 0 0 356230 233 0 0 20 0 3 0 4824952 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 -1 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
[pid=34369/tid=34374] ppid=34366 vsize=2386436 memory=12860 CPUtime=0.04 cores=1,3
/proc/34369/task/34374/stat : 34374 (fscip) S 34366 34369 30613 0 -1 1077936192 575 0 0 0 1 3 0 0 20 0 3 0 4825021 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 1 0 0 -1 3 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
Current children cumulated CPU time: 3570.41 s
Current children cumulated vsize: 2386436 KiB
Current children cumulated memory: 2199936 KiB

[startup+3599.27 s]
# the end of solver process 34369 was just reported to runsolver
# Child status: 0

# cumulated CPU time of all completed processes:  user=3584.51 s, system=2.81957 s

Solver just ended.
??? end of watcher thread
Dumping a history of the last processes samples

[startup+3584 s]*
/proc/loadavg: 4.08 4.14 4.11 5/267 36282
/proc/meminfo: memFree=109600688/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=2386436 memory=2199936 CPUtime=3572.1 cores=1,3
/proc/34369/stat : 34369 (fscip) S 34366 34369 30613 0 -1 1077936128 2955911 0 0 0 356932 278 0 0 20 0 3 0 4824414 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 596609 549984 2982 3035 0 585858 0
[pid=34369/tid=34373] ppid=34366 vsize=2386436 memory=-7235151017955910016 CPUtime=3566.33 cores=1,3
/proc/34369/task/34373/stat : 34373 (fscip) R 34366 34369 30613 0 -1 4194368 2689628 0 0 0 356400 233 0 0 20 0 3 0 4824952 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 -1 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
[pid=34369/tid=34374] ppid=34366 vsize=2386436 memory=7132 CPUtime=0.04 cores=1,3
/proc/34369/task/34374/stat : 34374 (fscip) S 34366 34369 30613 0 -1 1077936192 575 0 0 0 1 3 0 0 20 0 3 0 4825021 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 1 0 0 -1 3 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
Current children cumulated CPU time: 3572.1 s
Current children cumulated vsize: 2386436 KiB
Current children cumulated memory: 2199936 KiB

[startup+3590.4 s]
/proc/loadavg: 4.15 4.15 4.11 5/267 36282
/proc/meminfo: memFree=109600696/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=2386436 memory=2199936 CPUtime=3578.48 cores=1,3
/proc/34369/stat : 34369 (fscip) S 34366 34369 30613 0 -1 1077936128 2955926 0 0 0 357570 278 0 0 20 0 3 0 4824414 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 596609 549984 2982 3035 0 585858 0
[pid=34369/tid=34373] ppid=34366 vsize=2386436 memory=2199936 CPUtime=3572.7 cores=1,3
/proc/34369/task/34373/stat : 34373 (fscip) R 34366 34369 30613 0 -1 4194368 2689640 0 0 0 357037 233 0 0 20 0 3 0 4824952 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 -1 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
[pid=34369/tid=34374] ppid=34366 vsize=2386436 memory=2199936 CPUtime=0.04 cores=1,3
/proc/34369/task/34374/stat : 34374 (fscip) S 34366 34369 30613 0 -1 1077936192 578 0 0 0 1 3 0 0 20 0 3 0 4825021 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 1 0 0 -1 3 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
Current children cumulated CPU time: 3578.48 s
Current children cumulated vsize: 2386436 KiB
Current children cumulated memory: 2199936 KiB

[startup+3593.6 s]
/proc/loadavg: 4.14 4.15 4.11 5/267 36283
/proc/meminfo: memFree=109600556/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=2386436 memory=2199936 CPUtime=3581.67 cores=1,3
/proc/34369/stat : 34369 (fscip) S 34366 34369 30613 0 -1 1077936128 2955927 0 0 0 357889 278 0 0 20 0 3 0 4824414 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 596609 549984 2982 3035 0 585858 0
[pid=34369/tid=34373] ppid=34366 vsize=2386436 memory=2199936 CPUtime=3575.89 cores=1,3
/proc/34369/task/34373/stat : 34373 (fscip) R 34366 34369 30613 0 -1 4194368 2689641 0 0 0 357356 233 0 0 20 0 3 0 4824952 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 -1 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
[pid=34369/tid=34374] ppid=34366 vsize=2386436 memory=2199936 CPUtime=0.04 cores=1,3
/proc/34369/task/34374/stat : 34374 (fscip) S 34366 34369 30613 0 -1 1077936192 578 0 0 0 1 3 0 0 20 0 3 0 4825021 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 1 0 0 -1 3 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
Current children cumulated CPU time: 3581.67 s
Current children cumulated vsize: 2386436 KiB
Current children cumulated memory: 2199936 KiB

[startup+3596.8 s]
/proc/loadavg: 4.14 4.15 4.11 6/265 36329
/proc/meminfo: memFree=111541368/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=2386436 memory=2199936 CPUtime=3584.86 cores=1,3
/proc/34369/stat : 34369 (fscip) S 34366 34369 30613 0 -1 1077936128 2955935 0 0 0 358208 278 0 0 20 0 3 0 4824414 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 596609 549984 2982 3035 0 585858 0
[pid=34369/tid=34373] ppid=34366 vsize=2386436 memory=2199936 CPUtime=3579.08 cores=1,3
/proc/34369/task/34373/stat : 34373 (fscip) R 34366 34369 30613 0 -1 4194368 2689648 0 0 0 357675 233 0 0 20 0 3 0 4824952 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 -1 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
[pid=34369/tid=34374] ppid=34366 vsize=2386436 memory=2199936 CPUtime=0.04 cores=1,3
/proc/34369/task/34374/stat : 34374 (fscip) S 34366 34369 30613 0 -1 1077936192 579 0 0 0 1 3 0 0 20 0 3 0 4825021 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 1 0 0 -1 3 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
Current children cumulated CPU time: 3584.86 s
Current children cumulated vsize: 2386436 KiB
Current children cumulated memory: 2199936 KiB

[startup+3598.4 s]
/proc/loadavg: 4.13 4.14 4.11 5/267 36331
/proc/meminfo: memFree=111249336/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=2386436 memory=2199936 CPUtime=3586.45 cores=1,3
/proc/34369/stat : 34369 (fscip) S 34366 34369 30613 0 -1 1077936128 2959231 0 0 0 358367 278 0 0 20 0 3 0 4824414 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 3 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 596609 549984 2982 3035 0 585858 0
[pid=34369/tid=34373] ppid=34366 vsize=2386436 memory=12860 CPUtime=3580.67 cores=1,3
/proc/34369/task/34373/stat : 34373 (fscip) R 34366 34369 30613 0 -1 4194368 2692926 0 0 0 357834 233 0 0 20 0 3 0 4824952 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 -1 1 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
[pid=34369/tid=34374] ppid=34366 vsize=2386436 memory=13012 CPUtime=0.04 cores=1,3
/proc/34369/task/34374/stat : 34374 (fscip) S 34366 34369 30613 0 -1 1077936192 579 0 0 0 1 3 0 0 20 0 3 0 4825021 2443710464 549984 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 1 0 0 -1 3 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
Current children cumulated CPU time: 3586.45 s
Current children cumulated vsize: 2386436 KiB
Current children cumulated memory: 2199936 KiB

[startup+3598.8 s]
/proc/loadavg: 4.13 4.14 4.11 5/267 36331
/proc/meminfo: memFree=111249336/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=2021160 memory=1829744 CPUtime=3586.85 cores=1,3
/proc/34369/stat : 34369 (fscip) R 34366 34369 30613 0 -1 4194304 2964730 0 0 0 358406 279 0 0 20 0 2 0 4824414 2069667840 457436 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 3 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 505290 457436 3030 3035 0 495766 0
[pid=34369/tid=34374] ppid=34366 vsize=2021160 memory=2268 CPUtime=0.04 cores=1,3
/proc/34369/task/34374/stat : 34374 (fscip) S 34366 34369 30613 0 -1 1077936192 579 0 0 0 1 3 0 0 20 0 2 0 4825021 2069667840 457436 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 1 0 0 -1 3 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
Current children cumulated CPU time: 3586.85 s
Current children cumulated vsize: 2021160 KiB
Current children cumulated memory: 1829744 KiB

[startup+3599.2 s]
/proc/loadavg: 4.13 4.14 4.11 6/261 36357
/proc/meminfo: memFree=111950720/131524600 swapFree=131071996/131071996
[pid=34369] ppid=34366 vsize=1941976 memory=1751028 CPUtime=3587.25 cores=1,3
/proc/34369/stat : 34369 (fscip) R 34366 34369 30613 0 -1 4194304 2972500 0 0 0 358444 281 0 0 20 0 2 0 4824414 1988583424 437757 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 0 0 0 17 3 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
/proc/34369/statm: 485494 437757 3030 3035 0 475970 0
[pid=34369/tid=34374] ppid=34366 vsize=1941976 memory=26144 CPUtime=0.04 cores=1,3
/proc/34369/task/34374/stat : 34374 (fscip) S 34366 34369 30613 0 -1 1077936192 579 0 0 0 1 3 0 0 20 0 2 0 4825021 1988583424 437757 134679101440 4194304 16623229 140731946038560 0 0 0 0 0 2 1 0 0 -1 3 0 0 0 0 0 18838560 18947056 27475968 140731946046486 140731946046602 140731946046602 140731946053616 0
Current children cumulated CPU time: 3587.25 s
Current children cumulated vsize: 1941976 KiB
Current children cumulated memory: 1751028 KiB

Child status: 0

Real time (s): 3599.27
CPU time (s): 3587.33
CPU user time (s): 3584.51
CPU system time (s): 2.81957
CPU usage (%): 99.6682
Max. virtual memory (cumulated for all children) (KiB): 2386436
Max. memory (cumulated for all children) (KiB): 2199936

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 3584.51
system time used= 2.81957
maximum resident set size= 2199936
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2977376
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3649
involuntary context switches= 3723


# summary of solver processes directly reported to runsolver:
#   pid: 34369
#   total CPU time (s): 3587.33
#   total CPU user time (s): 3584.51
#   total CPU system time (s): 2.81957

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 7.88042 second user time and 18.2659 second system time

The end

Launcher Data 

Begin job on nodeC034 at 2024-07-31 01:30:01
IDJOB=4464181
IDBENCH=155358
IDSOLVER=3092
FILE ID=nodeC034/4464181-1722382201
RUNJOBID= nodeC034-1722378597-30625
SLURM_JOB_ID= 8249857
Free space on /tmp= 889980 MiB

SOLVER NAME= FiberSCIP 2024-07-30
BENCH NAME= PB24/normalized-PB11/OPT-LIN/heinz/normalized-stp3d.opb
COMMAND LINE= ./fscip settings/racing.prm BENCHNAME -qsol -t TIMEOUT -sth 1
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp --file-size-limit 102400 -w /tmp/evaluation-result-4464181-1722382201/watcher-4464181-1722382201 -o /tmp/evaluation-result-4464181-1722382201/solver-4464181-1722382201 -C 3600 -W 3700 -M 31000 --bin-var /tmp/runsolver-exch-nodeC034-1722378597-30625 --watchdog 3760  ./fscip settings/racing.prm HOME/instance-4464181-1722382201.opb -qsol -t 3600 -sth 1

TIME LIMIT= 3600 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 2

MD5SUM BENCH= a4b0cd988cdc47ea50384dbc6c682a43
RANDOM SEED=1078368009

nodeC034.alineos.net Linux 4.18.0-301.1.el8.x86_64 #1 SMP Tue Apr 13 16:24:22 UTC 2021

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3666.537
cache size	: 15360 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 6999.51
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3662.981
cache size	: 15360 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 7005.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3599.801
cache size	: 15360 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 6999.51
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3638.152
cache size	: 15360 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 7005.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3666.549
cache size	: 15360 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 6999.51
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3660.296
cache size	: 15360 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 7005.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3599.880
cache size	: 15360 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 6999.51
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
stepping	: 1
microcode	: 0xb00002a
cpu MHz		: 3617.148
cache size	: 15360 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 7005.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       131524600 kB
MemFree:        127124132 kB
MemAvailable:   127359260 kB
Buffers:           94184 kB
Cached:           974052 kB
SwapCached:            0 kB
Active:           274512 kB
Inactive:        3643552 kB
Active(anon):       1532 kB
Inactive(anon):  2869312 kB
Active(file):     272980 kB
Inactive(file):   774240 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:      131071996 kB
SwapFree:       131071996 kB
Dirty:             73568 kB
Writeback:             0 kB
AnonPages:       2847564 kB
Mapped:           124400 kB
Shmem:             21332 kB
KReclaimable:      55464 kB
Slab:             136392 kB
SReclaimable:      55464 kB
SUnreclaim:        80928 kB
KernelStack:        4244 kB
PageTables:        15628 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    196834296 kB
Committed_AS:    2025592 kB
VmallocTotal:   34359738367 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
Percpu:             9888 kB
HardwareCorrupted:     0 kB
AnonHugePages:   1366016 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
FileHugePages:         0 kB
FilePmdMapped:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB
DirectMap4k:      185044 kB
DirectMap2M:     5879808 kB
DirectMap1G:    130023424 kB

Free space on /tmp at the end= 889908 MiB
End job on nodeC034 at 2024-07-31 02:30:01