Trace number 4397880

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.0? (MO) 1520.94 1523.35

General information on the benchmark

NameSteelMillSlab/
SteelMillSlab-bench_3_12.xml
MD5SUM75a96ae289dcd0162feddbff51d0d425
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkSAT TO
Best value of the objective obtained on this benchmark1336
Best CPU time to get the best result obtained on this benchmark1997.76
Satisfiable
(Un)Satisfiability was proved
Number of variables12654
Number of constraints466
Number of domains4
Minimum domain size2
Maximum domain size111
Distribution of domain sizes[{"size":2,"count":12321},{"size":28,"count":111},{"size":38,"count":111},{"size":111,"count":111}]
Minimum variable degree2
Maximum variable degree114
Distribution of variable degrees[{"degree":2,"count":12432},{"degree":3,"count":111},{"degree":112,"count":78},{"degree":113,"count":24},{"degree":114,"count":9}]
Minimum constraint arity2
Maximum constraint arity112
Distribution of constraint arities[{"arity":2,"count":132},{"arity":111,"count":112},{"arity":112,"count":222}]
Number of extensional constraints0
Number of intensional constraints132
Distribution of constraint types[{"type":"intension","count":132},{"type":"sum","count":112},{"type":"element","count":111},{"type":"channel","count":111}]
Optimization problemYES
Type of objectivemin SUM

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (13765 MiB free)
  memory of node 1: 16384 MiB (15781 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4397880-1568589768/watcher-4397880-1568589768 -o /tmp/evaluation-result-4397880-1568589768/solver-4397880-1568589768 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node104-1568568567-1514 --watchdog 2560 cosoco -lc -model -restarts -stick HOME/instance-4397880-1568589768.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2500 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=5775, runsolver pid=5772

[startup+0.100157 s]*
/proc/loadavg: 2.00 2.01 2.05 3/199 5776
/proc/meminfo: memFree=30245072/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=39156 memory=15232 CPUtime=0.1 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 4008 0 0 0 10 0 0 0 20 0 1 0 435233928 40095744 3808 33554432000 4194304 6211092 140733203220944 140733203216104 140241533193112 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 9789 3808 809 493 0 3094 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 39156 KiB
Current children cumulated memory: 15232 KiB

[startup+0.209882 s]*
/proc/loadavg: 2.00 2.01 2.05 3/199 5776
/proc/meminfo: memFree=30243212/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=40728 memory=17000 CPUtime=0.2 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 4429 0 0 0 20 0 0 0 20 0 1 0 435233928 41705472 4250 33554432000 4194304 6211092 140733203220944 140733203216104 140241524179970 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 10182 4250 819 493 0 3487 0
Current children cumulated CPU time: 0.2 s
Current children cumulated vsize: 40728 KiB
Current children cumulated memory: 17000 KiB

[startup+0.309121 s]*
/proc/loadavg: 2.00 2.01 2.05 3/199 5776
/proc/meminfo: memFree=30242096/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=41716 memory=18056 CPUtime=0.3 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 4683 0 0 0 30 0 0 0 20 0 1 0 435233928 42717184 4514 33554432000 4194304 6211092 140733203220944 140733203194504 140241533201245 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 10429 4514 819 493 0 3734 0
Current children cumulated CPU time: 0.3 s
Current children cumulated vsize: 41716 KiB
Current children cumulated memory: 18056 KiB

[startup+0.700276 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 5776
/proc/meminfo: memFree=30240236/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=45712 memory=21992 CPUtime=0.69 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 5673 0 0 0 68 1 0 0 20 0 1 0 435233928 46809088 5498 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 11428 5498 819 493 0 4733 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 45712 KiB
Current children cumulated memory: 21992 KiB

[startup+1.50028 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 5776
/proc/meminfo: memFree=30235028/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=54080 memory=30052 CPUtime=1.49 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 7738 0 0 0 148 1 0 0 20 0 1 0 435233928 55377920 7513 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 13520 7513 819 493 0 6825 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 54080 KiB
Current children cumulated memory: 30052 KiB

[startup+3.10955 s]
/proc/loadavg: 2.08 2.03 2.05 3/199 5776
/proc/meminfo: memFree=30214612/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=70896 memory=46812 CPUtime=3.09 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 12061 0 0 0 308 1 0 0 20 0 1 0 435233928 72597504 11703 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 17724 11703 819 493 0 11029 0
Current children cumulated CPU time: 3.09 s
Current children cumulated vsize: 70896 KiB
Current children cumulated memory: 46812 KiB

[startup+6.30027 s]
/proc/loadavg: 2.08 2.03 2.05 3/199 5776
/proc/meminfo: memFree=30183748/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=104440 memory=79732 CPUtime=6.29 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 20467 0 0 0 626 3 0 0 20 0 1 0 435233928 106946560 19933 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 26110 19933 819 493 0 19415 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 104440 KiB
Current children cumulated memory: 79732 KiB

[startup+12.7003 s]
/proc/loadavg: 2.22 2.06 2.06 3/199 5776
/proc/meminfo: memFree=30121716/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=169900 memory=145728 CPUtime=12.69 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 37162 0 0 0 1263 6 0 0 20 0 1 0 435233928 173977600 36432 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 42475 36432 819 493 0 35780 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 169900 KiB
Current children cumulated memory: 145728 KiB

[startup+25.5003 s]
/proc/loadavg: 2.34 2.09 2.07 3/199 5777
/proc/meminfo: memFree=29986244/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=303340 memory=278940 CPUtime=25.48 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 70244 0 0 0 2539 9 0 0 20 0 1 0 435233928 310620160 69735 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 75835 69735 819 493 0 69140 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 303340 KiB
Current children cumulated memory: 278940 KiB

[startup+51.1095 s]
/proc/loadavg: 2.28 2.10 2.07 3/199 5779
/proc/meminfo: memFree=29716964/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=572100 memory=542892 CPUtime=51.07 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 135921 0 0 0 5090 17 0 0 20 0 1 0 435233928 585830400 135723 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 143025 135723 819 493 0 136330 0
Current children cumulated CPU time: 51.07 s
Current children cumulated vsize: 572100 KiB
Current children cumulated memory: 542892 KiB

[startup+102.3 s]
/proc/loadavg: 2.11 2.08 2.07 3/199 5783
/proc/meminfo: memFree=29186768/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=1113976 memory=1075928 CPUtime=102.23 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 265660 0 0 0 10187 36 0 0 20 0 1 0 435233928 1140711424 268982 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 278494 268982 819 493 0 271799 0
Current children cumulated CPU time: 102.23 s
Current children cumulated vsize: 1113976 KiB
Current children cumulated memory: 1075928 KiB

[startup+162.3 s]
/proc/loadavg: 2.04 2.07 2.06 3/198 5787
/proc/meminfo: memFree=28561336/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=1741688 memory=1697632 CPUtime=162.19 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 416959 0 0 0 16162 57 0 0 20 0 1 0 435233928 1783488512 424408 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 435422 424408 819 493 0 428727 0
Current children cumulated CPU time: 162.19 s
Current children cumulated vsize: 1741688 KiB
Current children cumulated memory: 1697632 KiB

[startup+222.3 s]
/proc/loadavg: 2.26 2.12 2.08 3/199 5854
/proc/meminfo: memFree=27776344/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=2344796 memory=2319268 CPUtime=222.15 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 567757 0 0 0 22138 77 0 0 20 0 1 0 435233928 2401071104 579817 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 586199 579817 819 493 0 579504 0
Current children cumulated CPU time: 222.15 s
Current children cumulated vsize: 2344796 KiB
Current children cumulated memory: 2319268 KiB

[startup+282.3 s]
/proc/loadavg: 2.10 2.10 2.07 3/198 5858
/proc/meminfo: memFree=26524668/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=2984948 memory=2942976 CPUtime=282.12 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 719145 0 0 0 28116 96 0 0 20 0 1 0 435233928 3056586752 735744 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 746237 735744 819 493 0 739542 0
Current children cumulated CPU time: 282.12 s
Current children cumulated vsize: 2984948 KiB
Current children cumulated memory: 2942976 KiB

[startup+342.3 s]
/proc/loadavg: 2.04 2.08 2.07 3/198 5862
/proc/meminfo: memFree=25275756/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=3643288 memory=3564172 CPUtime=342.08 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 870452 0 0 0 34091 117 0 0 20 0 1 0 435233928 3730726912 891043 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 910822 891043 819 493 0 904127 0
Current children cumulated CPU time: 342.08 s
Current children cumulated vsize: 3643288 KiB
Current children cumulated memory: 3564172 KiB

[startup+402.3 s]
/proc/loadavg: 2.01 2.07 2.06 3/199 5868
/proc/meminfo: memFree=24027132/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=4246264 memory=4187724 CPUtime=402.04 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 1021324 0 0 0 40067 137 0 0 20 0 1 0 435233928 4348174336 1046931 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 1061566 1046931 819 493 0 1054871 0
Current children cumulated CPU time: 402.04 s
Current children cumulated vsize: 4246264 KiB
Current children cumulated memory: 4187724 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 2.05 2.05 3/198 5879
/proc/meminfo: memFree=22777392/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=4849108 memory=4809080 CPUtime=462.01 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 1172187 0 0 0 46042 159 0 0 20 0 1 0 435233928 4965486592 1202270 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 1212277 1202270 819 493 0 1205582 0
Current children cumulated CPU time: 462.01 s
Current children cumulated vsize: 4849108 KiB
Current children cumulated memory: 4809080 KiB


################
# More data... #
################

Current children cumulated vsize: 9277024 KiB
Current children cumulated memory: 9166004 KiB

[startup+942.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 5919
/proc/meminfo: memFree=12785740/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=9880000 memory=9789236 CPUtime=941.8 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 2380911 0 0 0 93828 352 0 0 20 0 1 0 435233928 10117120000 2447309 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 2470000 2447309 819 493 0 2463305 0
Current children cumulated CPU time: 941.8 s
Current children cumulated vsize: 9880000 KiB
Current children cumulated memory: 9789236 KiB

[startup+1002.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/198 5923
/proc/meminfo: memFree=11537996/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=10482580 memory=10410308 CPUtime=1001.79 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 2531786 0 0 0 99799 380 0 0 20 0 1 0 435233928 10734161920 2602577 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 2620645 2602577 819 493 0 2613950 0
Current children cumulated CPU time: 1001.79 s
Current children cumulated vsize: 10482580 KiB
Current children cumulated memory: 10410308 KiB

[startup+1062.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 5935
/proc/meminfo: memFree=10289584/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=11085028 memory=11033316 CPUtime=1061.76 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 2682636 0 0 0 105771 405 0 0 20 0 1 0 435233928 11351068672 2758329 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 2771257 2758329 819 493 0 2764562 0
Current children cumulated CPU time: 1061.76 s
Current children cumulated vsize: 11085028 KiB
Current children cumulated memory: 11033316 KiB

[startup+1122.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 5941
/proc/meminfo: memFree=9041804/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=11687608 memory=11654456 CPUtime=1121.75 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 2833540 0 0 0 111741 434 0 0 20 0 1 0 435233928 11968110592 2913614 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 2921902 2913614 819 493 0 2915207 0
Current children cumulated CPU time: 1121.75 s
Current children cumulated vsize: 11687608 KiB
Current children cumulated memory: 11654456 KiB

[startup+1182.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 5945
/proc/meminfo: memFree=7795028/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=12476912 memory=12276560 CPUtime=1181.73 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 2984763 0 0 0 117710 463 0 0 20 0 1 0 435233928 12776357888 3069140 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 3119228 3069140 819 493 0 3112533 0
Current children cumulated CPU time: 1181.73 s
Current children cumulated vsize: 12476912 KiB
Current children cumulated memory: 12276560 KiB

[startup+1242.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/198 5949
/proc/meminfo: memFree=6549356/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=13079228 memory=12899520 CPUtime=1241.71 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 3135597 0 0 0 123677 494 0 0 20 0 1 0 435233928 13393129472 3224880 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 3269807 3224880 819 493 0 3263112 0
Current children cumulated CPU time: 1241.71 s
Current children cumulated vsize: 13079228 KiB
Current children cumulated memory: 12899520 KiB

[startup+1302.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 5956
/proc/meminfo: memFree=5300876/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=13681544 memory=13520180 CPUtime=1301.7 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 3286439 0 0 0 129645 525 0 0 20 0 1 0 435233928 14009901056 3380045 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 3420386 3380045 819 493 0 3413691 0
Current children cumulated CPU time: 1301.7 s
Current children cumulated vsize: 13681544 KiB
Current children cumulated memory: 13520180 KiB

[startup+1362.3 s]
/proc/loadavg: 2.08 2.03 2.05 3/199 5960
/proc/meminfo: memFree=4054072/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=14283860 memory=14140836 CPUtime=1361.68 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 3437260 0 0 0 135611 557 0 0 20 0 1 0 435233928 14626672640 3535209 33554432000 4194304 6211092 140733203220944 140733203194504 5012962 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 3570965 3535209 819 493 0 3564270 0
Current children cumulated CPU time: 1361.68 s
Current children cumulated vsize: 14283860 KiB
Current children cumulated memory: 14140836 KiB

[startup+1422.3 s]
/proc/loadavg: 2.44 2.16 2.09 3/198 5964
/proc/meminfo: memFree=2807700/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=14886044 memory=14763604 CPUtime=1421.65 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 3588103 0 0 0 141577 588 0 0 20 0 1 0 435233928 15243309056 3690901 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 3721511 3690901 819 493 0 3714816 0
Current children cumulated CPU time: 1421.65 s
Current children cumulated vsize: 14886044 KiB
Current children cumulated memory: 14763604 KiB

[startup+1482.3 s]
/proc/loadavg: 2.16 2.13 2.09 3/199 5970
/proc/meminfo: memFree=1560244/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=15487832 memory=15383952 CPUtime=1481.64 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 3738843 0 0 0 147540 624 0 0 20 0 1 0 435233928 15859539968 3845988 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 3871958 3845988 819 493 0 3865263 0
Current children cumulated CPU time: 1481.64 s
Current children cumulated vsize: 15487832 KiB
Current children cumulated memory: 15383952 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1520.6 s]
/proc/loadavg: 2.09 2.12 2.08 3/198 5970
/proc/meminfo: memFree=770076/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=15872216 memory=15782696 CPUtime=1519.93 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 4202496 3835114 0 0 0 151350 643 0 0 20 0 1 0 435233928 16253149184 3945674 33554432000 4194304 6211092 140733203220944 140733203194504 140241523256293 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 3968054 3945674 819 493 0 3961359 0
Current children cumulated CPU time: 1519.93 s
Current children cumulated vsize: 15872216 KiB
Current children cumulated memory: 15782696 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1523.35 s]
# the end of solver process 5775 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1513.74 s, system=7.19949 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1521.6 s]
/proc/loadavg: 2.09 2.12 2.08 3/199 5971
/proc/meminfo: memFree=534964/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=15872216 memory=15782696 CPUtime=1520.07 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) R 5772 5775 1482 0 -1 512 3835119 0 0 0 151363 644 0 0 20 0 1 0 435233928 16253149184 3945674 33554432000 4194304 6211092 140733203220944 140733203194504 5189768 0 16384 4096 8405026 18446744071580971376 0 0 17 3 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 3968054 3945674 819 493 0 3961359 0
Current children cumulated CPU time: 1520.07 s
Current children cumulated vsize: 15872216 KiB
Current children cumulated memory: 15782696 KiB

[startup+1522.4 s]
/proc/loadavg: 2.16 2.13 2.09 3/199 5971
/proc/meminfo: memFree=303452/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=15872216 memory=15782696 CPUtime=1520.15 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) S 5772 5775 1482 0 -1 512 3835119 0 0 0 151371 644 0 0 20 0 1 0 435233928 16253149184 3945674 33554432000 4194304 6211092 140733203220944 140733203194504 5189768 0 16384 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 3968054 3945674 819 493 0 3961359 0
Current children cumulated CPU time: 1520.15 s
Current children cumulated vsize: 15872216 KiB
Current children cumulated memory: 15782696 KiB

[startup+1522.6 s]
/proc/loadavg: 2.16 2.13 2.09 3/199 5971
/proc/meminfo: memFree=303452/32770624 swapFree=66913152/67108860
[pid=5775] ppid=5772 vsize=15872216 memory=15782696 CPUtime=1520.18 cores=1,3,5,7
/proc/5775/stat : 5775 (cosoco) S 5772 5775 1482 0 -1 512 3835119 0 0 0 151374 644 0 0 20 0 1 0 435233928 16253149184 3945674 33554432000 4194304 6211092 140733203220944 140733203194504 5189768 0 16384 4096 8405026 18446744071580971376 0 0 17 3 0 0 0 0 0 8312316 8317760 39927808 140733203227659 140733203227761 140733203227761 140733203230670 0
/proc/5775/statm: 3968054 3945674 819 493 0 3961359 0
Current children cumulated CPU time: 1520.18 s
Current children cumulated vsize: 15872216 KiB
Current children cumulated memory: 15782696 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1523.35
CPU time (s): 1520.94
CPU user time (s): 1513.74
CPU system time (s): 7.19949
CPU usage (%): 99.8418
Max. virtual memory (cumulated for all children) (KiB): 15872216
Max. memory (cumulated for all children) (KiB): 15782696

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1513.74
system time used= 7.19949
maximum resident set size= 15782716
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3835119
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 110603
involuntary context switches= 4719


# summary of solver processes directly reported to runsolver:
#   pid: 5775
#   total CPU time (s): 1520.94
#   total CPU user time (s): 1513.74
#   total CPU system time (s): 7.19949

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 5.49333 second user time and 10.3664 second system time

The end

Launcher Data 

Begin job on node104 at 2019-09-16 00:12:30
IDJOB=4397880
IDBENCH=139399
IDSOLVER=2850
FILE ID=node104/4397880-1568589768
RUNJOBID= node104-1568568567-1514
PBS_JOBID= 21703336
Free space on /tmp= 59008 MiB

SOLVER NAME= cosoco 2.0
BENCH NAME= XCSP17/SteelMillSlab/SteelMillSlab-bench_3_12.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4397880-1568589768/watcher-4397880-1568589768 -o /tmp/evaluation-result-4397880-1568589768/solver-4397880-1568589768 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node104-1568568567-1514 --watchdog 2560  cosoco -lc -model -restarts -stick HOME/instance-4397880-1568589768.xml

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 75a96ae289dcd0162feddbff51d0d425
RANDOM SEED=1080157230

node104.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        30256312 kB
MemAvailable:   31774228 kB
Buffers:          146436 kB
Cached:          1651040 kB
SwapCached:        95316 kB
Active:           721408 kB
Inactive:        1185340 kB
Active(anon):      18156 kB
Inactive(anon):   140320 kB
Active(file):     703252 kB
Inactive(file):  1045020 kB
Unevictable:      110868 kB
Mlocked:          110868 kB
SwapTotal:      67108860 kB
SwapFree:       66913152 kB
Dirty:              2760 kB
Writeback:             0 kB
AnonPages:        126656 kB
Mapped:            47736 kB
Shmem:             40588 kB
Slab:             200352 kB
SReclaimable:     170956 kB
SUnreclaim:        29396 kB
KernelStack:        3504 kB
PageTables:         5612 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     539532 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     79872 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      124288 kB
DirectMap2M:    33421312 kB

Free space on /tmp at the end= 58580 MiB
End job on node104 at 2019-09-16 00:37:53