Trace number 4390247

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2? (MO) 1679.91 1682.04

General information on the benchmark

NameOpd/Opd-sum-small/
Opd-10-037-014.xml
MD5SUMd77adff4d02ea0a040e7fe9c51d12b8d
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark5
Best CPU time to get the best result obtained on this benchmark2.75092
Satisfiable
(Un)Satisfiability was proved
Number of variables371
Number of constraints56
Number of domains2
Minimum domain size2
Maximum domain size33
Distribution of domain sizes[{"size":2,"count":370},{"size":33,"count":1}]
Minimum variable degree11
Maximum variable degree46
Distribution of variable degrees[{"degree":11,"count":370},{"degree":46,"count":1}]
Minimum constraint arity37
Maximum constraint arity370
Distribution of constraint arities[{"arity":37,"count":10},{"arity":75,"count":45},{"arity":370,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":55}]
Optimization problemYES
Type of objectivemin VAR

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (8637 MiB free)
  memory of node 1: 16384 MiB (11662 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4390247-1568335712/watcher-4390247-1568335712 -o /tmp/evaluation-result-4390247-1568335712/solver-4390247-1568335712 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node125-1568323660-10513 --watchdog 2560 cosoco -lc -model -restarts -stick HOME/instance-4390247-1568335712.xml 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2500 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=12567, runsolver pid=12564
Current StackSize limit: 8192 KiB


[startup+0.100196 s]*
/proc/loadavg: 2.00 2.00 2.04 3/199 12568
/proc/meminfo: memFree=20786980/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=28424 memory=4744 CPUtime=0.1 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 1409 0 0 0 10 0 0 0 20 0 1 0 409838095 29106176 1186 33554432000 4194304 6210612 140730865349984 140730865346520 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 7106 1186 844 493 0 411 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 28424 KiB
Current children cumulated memory: 4744 KiB

[startup+0.209421 s]*
/proc/loadavg: 2.00 2.00 2.04 3/199 12568
/proc/meminfo: memFree=20785864/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=29504 memory=5800 CPUtime=0.21 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 1665 0 0 0 21 0 0 0 20 0 1 0 409838095 30212096 1450 33554432000 4194304 6210612 140730865349984 140730865346520 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 7376 1450 845 493 0 681 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 29504 KiB
Current children cumulated memory: 5800 KiB

[startup+0.308884 s]*
/proc/loadavg: 2.00 2.00 2.04 3/199 12568
/proc/meminfo: memFree=20785088/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=30468 memory=6856 CPUtime=0.31 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 1897 0 0 0 31 0 0 0 20 0 1 0 409838095 31199232 1714 33554432000 4194304 6210612 140730865349984 140730865346520 5442833 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 7617 1714 845 493 0 922 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 30468 KiB
Current children cumulated memory: 6856 KiB

[startup+0.700339 s]
/proc/loadavg: 2.00 2.00 2.04 3/199 12568
/proc/meminfo: memFree=20783228/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=34108 memory=10404 CPUtime=0.69 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 2795 0 0 0 69 0 0 0 20 0 1 0 409838095 34926592 2601 33554432000 4194304 6210612 140730865349984 140730865346520 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 8527 2601 845 493 0 1832 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 34108 KiB
Current children cumulated memory: 10404 KiB

[startup+1.50034 s]
/proc/loadavg: 2.00 2.00 2.04 3/199 12568
/proc/meminfo: memFree=20778424/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=41712 memory=17944 CPUtime=1.49 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 4669 0 0 0 149 0 0 0 20 0 1 0 409838095 42713088 4486 33554432000 4194304 6210612 140730865349984 140730865329464 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 10428 4486 845 493 0 3733 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 41712 KiB
Current children cumulated memory: 17944 KiB

[startup+3.10944 s]
/proc/loadavg: 2.00 2.00 2.04 3/199 12568
/proc/meminfo: memFree=20759824/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=57168 memory=33012 CPUtime=3.1 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 8483 0 0 0 309 1 0 0 20 0 1 0 409838095 58540032 8253 33554432000 4194304 6210612 140730865349984 140730865329464 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 14292 8253 845 493 0 7597 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 57168 KiB
Current children cumulated memory: 33012 KiB

[startup+6.30033 s]
/proc/loadavg: 2.00 2.00 2.04 3/199 12568
/proc/meminfo: memFree=20732120/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=86640 memory=62920 CPUtime=6.28 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 16000 0 0 0 626 2 0 0 20 0 1 0 409838095 88719360 15730 33554432000 4194304 6210612 140730865349984 140730865329464 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 21660 15730 845 493 0 14965 0
Current children cumulated CPU time: 6.28 s
Current children cumulated vsize: 86640 KiB
Current children cumulated memory: 62920 KiB

[startup+12.7003 s]
/proc/loadavg: 2.00 2.00 2.04 3/199 12568
/proc/meminfo: memFree=20675536/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=147396 memory=122672 CPUtime=12.69 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 31026 0 0 0 1265 4 0 0 20 0 1 0 409838095 150933504 30668 33554432000 4194304 6210612 140730865349984 140730865329464 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 36849 30668 845 493 0 30154 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 147396 KiB
Current children cumulated memory: 122672 KiB

[startup+25.5003 s]
/proc/loadavg: 2.00 2.00 2.04 3/199 12570
/proc/meminfo: memFree=20553848/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=269940 memory=242528 CPUtime=25.48 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 60540 0 0 0 2540 8 0 0 20 0 1 0 409838095 276418560 60632 33554432000 4194304 6210612 140730865349984 140730865329464 5012488 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 67485 60632 845 493 0 60790 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 269940 KiB
Current children cumulated memory: 242528 KiB

[startup+51.1094 s]
/proc/loadavg: 2.00 2.00 2.04 3/199 12570
/proc/meminfo: memFree=20310000/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=506252 memory=482516 CPUtime=51.08 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 118996 0 0 0 5093 15 0 0 20 0 1 0 409838095 518402048 120629 33554432000 4194304 6210612 140730865349984 140730865329464 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 126563 120629 845 493 0 119868 0
Current children cumulated CPU time: 51.08 s
Current children cumulated vsize: 506252 KiB
Current children cumulated memory: 482516 KiB

[startup+102.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/199 12574
/proc/meminfo: memFree=19829268/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=991368 memory=963804 CPUtime=102.26 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 234268 0 0 0 10198 28 0 0 20 0 1 0 409838095 1015160832 240951 33554432000 4194304 6210612 140730865349984 140730865329464 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 247842 240951 845 493 0 241147 0
Current children cumulated CPU time: 102.26 s
Current children cumulated vsize: 991368 KiB
Current children cumulated memory: 963804 KiB

[startup+162.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/199 12587
/proc/meminfo: memFree=19266440/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=1552552 memory=1525088 CPUtime=162.24 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 368955 0 0 0 16180 44 0 0 20 0 1 0 409838095 1589813248 381272 33554432000 4194304 6210612 140730865349984 140730865329464 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 388138 381272 845 493 0 381443 0
Current children cumulated CPU time: 162.24 s
Current children cumulated vsize: 1552552 KiB
Current children cumulated memory: 1525088 KiB

[startup+222.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/199 12591
/proc/meminfo: memFree=18704644/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=2126044 memory=2088468 CPUtime=222.24 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 503653 0 0 0 22163 61 0 0 20 0 1 0 409838095 2177069056 522117 33554432000 4194304 6210612 140730865349984 140730865329464 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 531511 522117 845 493 0 524816 0
Current children cumulated CPU time: 222.24 s
Current children cumulated vsize: 2126044 KiB
Current children cumulated memory: 2088468 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/200 12596
/proc/meminfo: memFree=18140576/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=2717988 memory=2649368 CPUtime=282.22 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 638340 0 0 0 28144 78 0 0 20 0 1 0 409838095 2783219712 662342 33554432000 4194304 6210612 140730865349984 140730865329464 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 679497 662342 845 493 0 672802 0
Current children cumulated CPU time: 282.22 s
Current children cumulated vsize: 2717988 KiB
Current children cumulated memory: 2649368 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/200 12602
/proc/meminfo: memFree=17577916/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=3254324 memory=3210460 CPUtime=342.21 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 772486 0 0 0 34125 96 0 0 20 0 1 0 409838095 3332427776 802615 33554432000 4194304 6210612 140730865349984 140730865329464 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 813581 802615 845 493 0 806886 0
Current children cumulated CPU time: 342.21 s
Current children cumulated vsize: 3254324 KiB
Current children cumulated memory: 3210460 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/200 12606
/proc/meminfo: memFree=17013860/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=3873736 memory=3772748 CPUtime=402.19 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 907474 0 0 0 40105 114 0 0 20 0 1 0 409838095 3966705664 943187 33554432000 4194304 6210612 140730865349984 140730865329464 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 968434 943187 845 493 0 961739 0
Current children cumulated CPU time: 402.19 s
Current children cumulated vsize: 3873736 KiB
Current children cumulated memory: 3772748 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/200 12611
/proc/meminfo: memFree=16449660/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=4410052 memory=4335748 CPUtime=462.18 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 1041662 0 0 0 46088 130 0 0 20 0 1 0 409838095 4515893248 1083937 33554432000 4194304 6210612 140730865349984 140730865329464 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 1102513 1083937 845 493 0 1095818 0
Current children cumulated CPU time: 462.18 s
Current children cumulated vsize: 4410052 KiB
Current children cumulated memory: 4335748 KiB


################
# More data... #
################

Current children cumulated vsize: 10622200 KiB
Current children cumulated memory: 10516268 KiB

[startup+1182.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/200 12759
/proc/meminfo: memFree=9693192/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=11157592 memory=11078268 CPUtime=1182.02 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 2652862 0 0 0 117815 387 0 0 20 0 1 0 409838095 11425374208 2769567 33554432000 4194304 6210612 140730865349984 140730865329464 140237869964094 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 2789398 2769567 845 493 0 2782703 0
Current children cumulated CPU time: 1182.02 s
Current children cumulated vsize: 11157592 KiB
Current children cumulated memory: 11078268 KiB

[startup+1242.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/200 12765
/proc/meminfo: memFree=9131128/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=11693504 memory=11638708 CPUtime=1242.01 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 2786992 0 0 0 123788 413 0 0 20 0 1 0 409838095 11974148096 2909677 33554432000 4194304 6210612 140730865349984 140730865329464 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 2923376 2909677 845 493 0 2916681 0
Current children cumulated CPU time: 1242.01 s
Current children cumulated vsize: 11693504 KiB
Current children cumulated memory: 11638708 KiB

[startup+1302.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/200 12769
/proc/meminfo: memFree=8569444/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=12229880 memory=12199492 CPUtime=1301.99 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 2921207 0 0 0 129757 442 0 0 20 0 1 0 409838095 12523397120 3049873 33554432000 4194304 6210612 140730865349984 140730865329464 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 3057470 3049873 845 493 0 3050775 0
Current children cumulated CPU time: 1301.99 s
Current children cumulated vsize: 12229880 KiB
Current children cumulated memory: 12199492 KiB

[startup+1362.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/200 12778
/proc/meminfo: memFree=8005864/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=13045552 memory=12761816 CPUtime=1361.98 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 3055722 0 0 0 135725 473 0 0 20 0 1 0 409838095 13358645248 3190454 33554432000 4194304 6210612 140730865349984 140730865329464 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 3261388 3190454 845 493 0 3254693 0
Current children cumulated CPU time: 1361.98 s
Current children cumulated vsize: 13045552 KiB
Current children cumulated memory: 12761816 KiB

[startup+1422.3 s]
/proc/loadavg: 2.06 2.03 2.05 3/200 12784
/proc/meminfo: memFree=7445052/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=13580812 memory=13321388 CPUtime=1421.96 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 3189690 0 0 0 141691 505 0 0 20 0 1 0 409838095 13906751488 3330347 33554432000 4194304 6210612 140730865349984 140730865329464 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 3395203 3330347 845 493 0 3388508 0
Current children cumulated CPU time: 1421.96 s
Current children cumulated vsize: 13580812 KiB
Current children cumulated memory: 13321388 KiB

[startup+1482.3 s]
/proc/loadavg: 2.02 2.02 2.05 3/200 12788
/proc/meminfo: memFree=6882072/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=14116072 memory=13883372 CPUtime=1481.96 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 3323641 0 0 0 147663 533 0 0 20 0 1 0 409838095 14454857728 3470843 33554432000 4194304 6210612 140730865349984 140730865329464 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 3529018 3470843 845 493 0 3522323 0
Current children cumulated CPU time: 1481.96 s
Current children cumulated vsize: 14116072 KiB
Current children cumulated memory: 13883372 KiB

[startup+1542.3 s]
/proc/loadavg: 2.01 2.02 2.05 3/200 12792
/proc/meminfo: memFree=6320980/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=14651200 memory=14443180 CPUtime=1541.94 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 3457608 0 0 0 153628 566 0 0 20 0 1 0 409838095 15002828800 3610795 33554432000 4194304 6210612 140730865349984 140730865329464 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 3662800 3610795 845 493 0 3656105 0
Current children cumulated CPU time: 1541.94 s
Current children cumulated vsize: 14651200 KiB
Current children cumulated memory: 14443180 KiB

[startup+1602.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 12798
/proc/meminfo: memFree=5758672/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=15186196 memory=15004764 CPUtime=1601.92 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 3591535 0 0 0 159593 599 0 0 20 0 1 0 409838095 15550664704 3751191 33554432000 4194304 6210612 140730865349984 140730865329464 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 3796549 3751191 845 493 0 3789854 0
Current children cumulated CPU time: 1601.92 s
Current children cumulated vsize: 15186196 KiB
Current children cumulated memory: 15004764 KiB

[startup+1662.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 12802
/proc/meminfo: memFree=5197700/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=15721192 memory=15564308 CPUtime=1661.91 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 3725453 0 0 0 165558 633 0 0 20 0 1 0 409838095 16098500608 3891077 33554432000 4194304 6210612 140730865349984 140730865329464 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 3930298 3891077 845 493 0 3923603 0
Current children cumulated CPU time: 1661.91 s
Current children cumulated vsize: 15721192 KiB
Current children cumulated memory: 15564308 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1679.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 12803
/proc/meminfo: memFree=5037624/32770624 swapFree=66939772/67108860
[pid=12567] ppid=12564 vsize=15872596 memory=15723860 CPUtime=1678.91 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 4202496 3763366 0 0 0 167246 645 0 0 20 0 1 0 409838095 16253538304 3930965 33554432000 4194304 6210612 140730865349984 140730865329464 140237869968357 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 3968149 3930965 845 493 0 3961454 0
Current children cumulated CPU time: 1678.91 s
Current children cumulated vsize: 15872596 KiB
Current children cumulated memory: 15723860 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1682.04 s]
# the end of solver process 12567 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1672.69 s, system=7.21956 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1680 s]*
/proc/loadavg: 2.00 2.01 2.05 3/201 12804
/proc/meminfo: memFree=4733328/32770624 swapFree=66939776/67108860
[pid=12567] ppid=12564 vsize=15872596 memory=15723860 CPUtime=1678.99 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) S 12564 12567 10482 0 -1 512 3763371 0 0 0 167253 646 0 0 20 0 1 0 409838095 16253538304 3930965 33554432000 4194304 6210612 140730865349984 140730865329464 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 3968149 3930965 845 493 0 3961454 0
Current children cumulated CPU time: 1678.99 s
Current children cumulated vsize: 15872596 KiB
Current children cumulated memory: 15723860 KiB

[startup+1680.8 s]
/proc/loadavg: 2.00 2.01 2.05 3/201 12804
/proc/meminfo: memFree=4733328/32770624 swapFree=66939776/67108860
[pid=12567] ppid=12564 vsize=15872596 memory=15723860 CPUtime=1679.1 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) R 12564 12567 10482 0 -1 512 3763371 0 0 0 167263 647 0 0 20 0 1 0 409838095 16253538304 3930965 33554432000 4194304 6210612 140730865349984 140730865329464 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 3968149 3930965 845 493 0 3961454 0
Current children cumulated CPU time: 1679.1 s
Current children cumulated vsize: 15872596 KiB
Current children cumulated memory: 15723860 KiB

[startup+1681.2 s]
/proc/loadavg: 2.00 2.01 2.05 3/202 12806
/proc/meminfo: memFree=4237972/32770624 swapFree=66939776/67108860
[pid=12567] ppid=12564 vsize=15872596 memory=15723860 CPUtime=1679.16 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) S 12564 12567 10482 0 -1 512 3763371 0 0 0 167268 648 0 0 20 0 1 0 409838095 16253538304 3930965 33554432000 4194304 6210612 140730865349984 140730865329464 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 3968149 3930965 845 493 0 3961454 0
Current children cumulated CPU time: 1679.16 s
Current children cumulated vsize: 15872596 KiB
Current children cumulated memory: 15723860 KiB

[startup+1681.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/202 12806
/proc/meminfo: memFree=4237972/32770624 swapFree=66939776/67108860
[pid=12567] ppid=12564 vsize=15872596 memory=15723860 CPUtime=1679.17 cores=0,2,4,6
/proc/12567/stat : 12567 (cosoco) S 12564 12567 10482 0 -1 512 3763371 0 0 0 167269 648 0 0 20 0 1 0 409838095 16253538304 3930965 33554432000 4194304 6210612 140730865349984 140730865329464 5188952 0 16384 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 18440192 140730865353739 140730865353841 140730865353841 140730865356750 0
/proc/12567/statm: 3968149 3930965 845 493 0 3961454 0
Current children cumulated CPU time: 1679.17 s
Current children cumulated vsize: 15872596 KiB
Current children cumulated memory: 15723860 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1682.04
CPU time (s): 1679.91
CPU user time (s): 1672.69
CPU system time (s): 7.21956
CPU usage (%): 99.8737
Max. virtual memory (cumulated for all children) (KiB): 15872596
Max. memory (cumulated for all children) (KiB): 15723860

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1672.69
system time used= 7.21956
maximum resident set size= 15724112
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3763371
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 116938
involuntary context switches= 2101


# summary of solver processes directly reported to runsolver:
#   pid: 12567
#   total CPU time (s): 1679.91
#   total CPU user time (s): 1672.69
#   total CPU system time (s): 7.21956

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 6.26266 second user time and 11.5409 second system time

The end

Launcher Data 

Begin job on node125 at 2019-09-13 02:47:50
IDJOB=4390247
IDBENCH=122974
IDSOLVER=2831
FILE ID=node125/4390247-1568335712
RUNJOBID= node125-1568323660-10513
PBS_JOBID= 21701146
Free space on /tmp= 57376 MiB

SOLVER NAME= cosoco 2
BENCH NAME= XCSP17/Opd/Opd-sum-small/Opd-10-037-014.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4390247-1568335712/watcher-4390247-1568335712 -o /tmp/evaluation-result-4390247-1568335712/solver-4390247-1568335712 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node125-1568323660-10513 --watchdog 2560  cosoco -lc -model -restarts -stick HOME/instance-4390247-1568335712.xml

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= d77adff4d02ea0a040e7fe9c51d12b8d
RANDOM SEED=1810561995

node125.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.29
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.29
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.29
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.29
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        20788024 kB
MemAvailable:   31794652 kB
Buffers:          186968 kB
Cached:         11000936 kB
SwapCached:        66612 kB
Active:           964768 kB
Inactive:       10293612 kB
Active(anon):       8208 kB
Inactive(anon):   103144 kB
Active(file):     956560 kB
Inactive(file): 10190468 kB
Unevictable:      108984 kB
Mlocked:          108984 kB
SwapTotal:      67108860 kB
SwapFree:       66939772 kB
Dirty:              2508 kB
Writeback:             0 kB
AnonPages:        255812 kB
Mapped:            44280 kB
Shmem:             32192 kB
Slab:             316852 kB
SReclaimable:     288060 kB
SUnreclaim:        28792 kB
KernelStack:        3760 kB
PageTables:         5852 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     528496 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:    208896 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      114048 kB
DirectMap2M:    33431552 kB

Free space on /tmp at the end= 56924 MiB
End job on node125 at 2019-09-13 03:15:52