Trace number 3386752

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-seq-random (fixed)? (TO) 5000.02 4982.2

General information on the benchmark

Namerandom/medium/
unif-k7-r89-v70-c6230-S1475900283-041.UNKNOWN.cnf
MD5SUMa83cf27d1d664cde32ad2f1a49b00933
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark4321.31
Satisfiable
(Un)Satisfiability was proved
Number of variables70
Number of clauses6230
Sum of the clauses size43610
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 56230

Solver Data 

0.31/0.45	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.31/0.45	c INFO: solving HOME/instance-3386752-1305591373.cnf
0.31/0.45	c INFO: computing task features
0.31/0.49	c INFO: computed variable-clause graph statistics
0.40/0.50	c INFO: computed clause balance statistics
0.60/0.71	c INFO: computed variable balance statistics
0.70/0.80	c INFO: computed small-clause counts
0.70/0.80	c INFO: computed Horn-clause counts
0.70/0.81	c INFO: computed variable graph statistics
0.70/0.81	c INFO: collected features for HOME/instance-3386752-1305591373.cnf in 0.35 s
0.70/0.84	c INFO: running gnovelty+2@0 for 97 with 4999 remaining (b = 0.52)
99.43/99.22	c INFO: running gnovelty+2@0 for 97 with 4901 remaining (b = 0.10)
198.14/197.60	c INFO: running adaptg2wsat2009++@0 for 97 with 4802 remaining (b = 0.05)
297.73/296.97	c INFO: running adaptg2wsat2009++@0 for 97 with 4703 remaining (b = 0.04)
398.37/397.33	c INFO: running adaptg2wsat2009++@0 for 97 with 4603 remaining (b = 0.03)
498.11/496.71	c INFO: running gnovelty+2@0 for 97 with 4503 remaining (b = 0.02)
598.70/597.08	c INFO: running FH@0 for 97 with 4403 remaining (b = 0.02)
698.45/696.47	c INFO: running march_hi@0 for 388 with 4304 remaining (b = 0.07)
1090.54/1087.05	c INFO: running march_hi@390 for 97 with 3913 remaining (b = 0.03)
1186.83/1182.90	c INFO: running march_hi@486 for 194 with 3817 remaining (b = 0.05)
1384.24/1379.66	c INFO: running march_hi@683 for 97 with 3621 remaining (b = 0.02)
1483.52/1478.54	c INFO: running adaptg2wsat2009++@0 for 97 with 3522 remaining (b = 0.02)
1584.20/1578.93	c INFO: running march_hi@0 for 388 with 3421 remaining (b = 0.07)
1978.06/1971.38	c INFO: running march_hi@392 for 97 with 3029 remaining (b = 0.03)
2074.91/2067.88	c INFO: running march_hi@488 for 97 with 2932 remaining (b = 0.02)
2169.38/2161.96	c INFO: running march_hi@582 for 97 with 2838 remaining (b = 0.03)
2271.35/2263.58	c INFO: running march_hi@0 for 388 with 2737 remaining (b = 0.06)
2662.32/2653.09	c INFO: running march_hi@389 for 97 with 2347 remaining (b = 0.02)
2758.51/2748.86	c INFO: running march_hi@485 for 194 with 2251 remaining (b = 0.03)
2956.86/2946.48	c INFO: running adaptg2wsat2009++@99 for 97 with 2054 remaining (b = 0.01)
3053.62/3042.87	c INFO: running FH@0 for 97 with 1957 remaining (b = 0.01)
3153.35/3142.26	c INFO: running hybridGM3@0 for 194 with 1858 remaining (b = 0.03)
3349.85/3338.02	c INFO: running gnovelty+2@0 for 97 with 1662 remaining (b = 0.01)
3450.48/3438.43	c INFO: running march_hi@781 for 1358 with 1562 remaining (b = 0.16)
4813.00/4795.85	c INFO: running march_hi@2139 for 97 with 205 remaining (b = 0.04)
4907.45/4889.94	c INFO: running march_hi@684 for 111 with 111 remaining (b = 0.01)
5000.02/4982.18	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3386752-1305591373/watcher-3386752-1305591373 -o /tmp/evaluation-result-3386752-1305591373/solver-3386752-1305591373 -C 5000 -W 5100 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3386752-1305591373.cnf -seed 779752522 -budget 5000 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 5030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.08 4.08 4.02 4/458 28874
/proc/meminfo: memFree=13990648/32950928 swapFree=62427640/67111528
[pid=28874] ppid=28872 vsize=16248 CPUtime=0 cores=1,3,5,7
/proc/28874/stat : 28874 (python) R 28872 28874 28027 0 -1 4202496 1051 0 0 0 0 0 0 0 25 0 1 0 670737152 16637952 923 33554432000 4194304 5683644 140734205892368 18446744073709551615 248089698048 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/28874/statm: 4062 923 381 364 0 549 0

[startup+0.093644 s]
/proc/loadavg: 4.08 4.08 4.02 4/458 28874
/proc/meminfo: memFree=13990648/32950928 swapFree=62427640/67111528
[pid=28874] ppid=28872 vsize=101880 CPUtime=0.06 cores=1,3,5,7
/proc/28874/stat : 28874 (python) R 28872 28874 28027 0 -1 4202496 2960 0 0 0 5 1 0 0 17 0 1 0 670737152 104325120 2654 33554432000 4194304 5683644 140734205892368 18446744073709551615 248089696256 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/28874/statm: 25470 2654 841 364 0 1770 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 101880

[startup+0.100637 s]
/proc/loadavg: 4.08 4.08 4.02 4/458 28874
/proc/meminfo: memFree=13990648/32950928 swapFree=62427640/67111528
[pid=28874] ppid=28872 vsize=104624 CPUtime=0.06 cores=1,3,5,7
/proc/28874/stat : 28874 (python) R 28872 28874 28027 0 -1 4202496 3119 0 0 0 5 1 0 0 17 0 1 0 670737152 107134976 2799 33554432000 4194304 5683644 140734205892368 18446744073709551615 4540219 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/28874/statm: 26156 2801 847 364 0 1904 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 104624

[startup+0.308642 s]
/proc/loadavg: 4.08 4.08 4.02 4/458 28874
/proc/meminfo: memFree=13990648/32950928 swapFree=62427640/67111528
[pid=28874] ppid=28872 vsize=198920 CPUtime=0.23 cores=1,3,5,7
/proc/28874/stat : 28874 (python) R 28872 28874 28027 0 -1 4202496 7028 933 1 0 18 5 0 0 17 0 1 0 670737152 203694080 5053 33554432000 4194304 5683644 140734205892368 18446744073709551615 4620076 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/28874/statm: 49730 5053 1466 364 0 3757 0
Current children cumulated CPU time (s) 0.23
Current children cumulated vsize (KiB) 198920

[startup+0.700533 s]
/proc/loadavg: 4.08 4.08 4.02 4/458 28874
/proc/meminfo: memFree=13990648/32950928 swapFree=62427640/67111528
[pid=28874] ppid=28872 vsize=279852 CPUtime=0.6 cores=1,3,5,7
/proc/28874/stat : 28874 (python) R 28872 28874 28027 0 -1 4202496 9778 933 1 0 53 7 0 0 17 0 1 0 670737152 286568448 7347 33554432000 4194304 5683644 140734205892368 18446744073709551615 248089356513 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/28874/statm: 69963 7347 1826 364 0 5887 0
Current children cumulated CPU time (s) 0.6
Current children cumulated vsize (KiB) 279852

[startup+1.50039 s]
/proc/loadavg: 4.08 4.08 4.02 5/462 28882
/proc/meminfo: memFree=13964328/32950928 swapFree=62427640/67111528
[pid=28874] ppid=28872 vsize=290120 CPUtime=0.74 cores=1,3,5,7
/proc/28874/stat : 28874 (python) S 28872 28874 28027 0 -1 4202496 10089 933 1 0 67 7 0 0 20 0 2 0 670737152 297082880 7408 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/28874/statm: 72530 7408 1877 364 0 8448 0
[pid=28880] ppid=28874 vsize=279876 CPUtime=0 cores=1,3,5,7
/proc/28880/stat : 28880 (python) S 28874 28874 28027 0 -1 4202560 743 0 0 0 0 0 0 0 19 0 1 0 670737236 286593024 5881 33554432000 4194304 5683644 140734205892368 18446744073709551615 248089718287 0 0 16781312 514 4460565021432217599 0 0 17 3 0 0 0
/proc/28880/statm: 69969 5881 354 364 0 5887 0
[pid=28882] ppid=28880 vsize=1018076 CPUtime=0.64 cores=1,3,5,7
/proc/28882/stat : 28882 (gnovelty+2) R 28880 28882 28882 0 -1 4202496 531 0 0 0 64 0 0 0 22 0 1 0 670737236 1042509824 267 33554432000 134512640 135663704 4291542800 18446744073709551615 134524882 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/28882/statm: 254519 267 113 282 0 254235 0
[pid=28874/tid=28881] ppid=28872 vsize=290120 CPUtime=0 cores=1,3,5,7
/proc/28874/task/28881/stat : 28881 (python) S 28872 28874 28027 0 -1 4202560 31 933 0 0 0 0 0 0 20 0 2 0 670737236 297082880 7408 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 2 18446604457759544576 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 1.38
Current children cumulated vsize (KiB) 1588072

[startup+3.10105 s]
/proc/loadavg: 4.08 4.08 4.02 5/462 28882
/proc/meminfo: memFree=13963212/32950928 swapFree=62427640/67111528
[pid=28874] ppid=28872 vsize=290120 CPUtime=0.74 cores=1,3,5,7
/proc/28874/stat : 28874 (python) S 28872 28874 28027 0 -1 4202496 10089 933 1 0 67 7 0 0 20 0 2 0 670737152 297082880 7408 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/28874/statm: 72530 7408 1877 364 0 8448 0
[pid=28880] ppid=28874 vsize=279876 CPUtime=0 cores=1,3,5,7
/proc/28880/stat : 28880 (python) S 28874 28874 28027 0 -1 4202560 743 0 0 0 0 0 0 0 18 0 1 0 670737236 286593024 5881 33554432000 4194304 5683644 140734205892368 18446744073709551615 248089718287 0 0 16781312 514 4460565021432185095 0 0 17 3 0 0 0
/proc/28880/statm: 69969 5881 354 364 0 5887 0
[pid=28882] ppid=28880 vsize=1018076 CPUtime=2.25 cores=1,3,5,7
/proc/28882/stat : 28882 (gnovelty+2) R 28880 28882 28882 0 -1 4202496 531 0 0 0 225 0 0 0 25 0 1 0 670737236 1042509824 267 33554432000 134512640 135663704 4291542800 18446744073709551615 134524882 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/28882/statm: 254519 267 113 282 0 254235 0
[pid=28874/tid=28881] ppid=28872 vsize=290120 CPUtime=0 cores=1,3,5,7
/proc/28874/task/28881/stat : 28881 (python) S 28872 28874 28027 0 -1 4202560 31 933 0 0 0 0 0 0 20 0 2 0 670737236 297082880 7408 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 2 18446604457759544576 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2.99
Current children cumulated vsize (KiB) 1588072

[startup+6.30054 s]
/proc/loadavg: 4.08 4.08 4.02 5/462 28882
/proc/meminfo: memFree=13962556/32950928 swapFree=62427640/67111528
[pid=28874] ppid=28872 vsize=290120 CPUtime=0.74 cores=1,3,5,7
/proc/28874/stat : 28874 (python) S 28872 28874 28027 0 -1 4202496 10089 933 1 0 67 7 0 0 20 0 2 0 670737152 297082880 7408 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/28874/statm: 72530 7408 1877 364 0 8448 0
[pid=28880] ppid=28874 vsize=279876 CPUtime=0.01 cores=1,3,5,7
/proc/28880/stat : 28880 (python) S 28874 28874 28027 0 -1 4202560 895 0 0 0 0 1 0 0 18 0 1 0 670737236 286593024 5904 33554432000 4194304 5683644 140734205892368 18446744073709551615 248089718287 0 0 16781312 514 4460565021432185095 0 0 17 7 0 0 0
/proc/28880/statm: 69969 5904 377 364 0 5887 0
[pid=28882] ppid=28880 vsize=1018076 CPUtime=5.45 cores=1,3,5,7
/proc/28882/stat : 28882 (gnovelty+2) R 28880 28882 28882 0 -1 4202496 531 0 0 0 545 0 0 0 25 0 1 0 670737236 1042509824 267 33554432000 134512640 135663704 4291542800 18446744073709551615 134525544 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/28882/statm: 254519 267 113 282 0 254235 0
[pid=28874/tid=28881] ppid=28872 vsize=290120 CPUtime=0 cores=1,3,5,7
/proc/28874/task/28881/stat : 28881 (python) S 28872 28874 28027 0 -1 4202560 31 933 0 0 0 0 0 0 20 0 2 0 670737236 297082880 7408 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 2 18446604457759544576 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 6.2
Current children cumulated vsize (KiB) 1588072

[startup+12.7004 s]
/proc/loadavg: 4.07 4.08 4.02 5/462 28882
/proc/meminfo: memFree=13961808/32950928 swapFree=62427640/67111528
[pid=28874] ppid=28872 vsize=290120 CPUtime=0.74 cores=1,3,5,7
/proc/28874/stat : 28874 (python) S 28872 28874 28027 0 -1 4202496 10089 933 1 0 67 7 0 0 20 0 2 0 670737152 297082880 7408 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/28874/statm: 72530 7408 1877 364 0 8448 0
[pid=28880] ppid=28874 vsize=279876 CPUtime=0.03 cores=1,3,5,7
/proc/28880/stat : 28880 (python) S 28874 28874 28027 0 -1 4202560 897 0 0 0 2 1 0 0 18 0 1 0 670737236 286593024 5904 33554432000 4194304 5683644 140734205892368 18446744073709551615 248089718287 0 0 16781312 514 15279112646738018567 0 0 17 7 0 0 0
/proc/28880/statm: 69969 5904 377 364 0 5887 0
[pid=28882] ppid=28880 vsize=1018076 CPUtime=11.85 cores=1,3,5,7
/proc/28882/stat : 28882 (gnovelty+2) R 28880 28882 28882 0 -1 4202496 531 0 0 0 1185 0 0 0 25 0 1 0 670737236 1042509824 267 33554432000 134512640 135663704 4291542800 18446744073709551615 134524868 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/28882/statm: 254519 267 113 282 0 254235 0
[pid=28874/tid=28881] ppid=28872 vsize=290120 CPUtime=0 cores=1,3,5,7
/proc/28874/task/28881/stat : 28881 (python) S 28872 28874 28027 0 -1 4202560 31 933 0 0 0 0 0 0 20 0 2 0 670737236 297082880 7408 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 2 18446604457759544576 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 12.62
Current children cumulated vsize (KiB) 1588072

[startup+25.5011 s]
/proc/loadavg: 4.05 4.08 4.02 5/462 28883
/proc/meminfo: memFree=13961196/32950928 swapFree=62427640/67111528
[pid=28874] ppid=28872 vsize=290120 CPUtime=0.74 cores=1,3,5,7
/proc/28874/stat : 28874 (python) S 28872 28874 28027 0 -1 4202496 10089 933 1 0 67 7 0 0 20 0 2 0 670737152 297082880 7408 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/28874/statm: 72530 7408 1877 364 0 8448 0
[pid=28880] ppid=28874 vsize=279876 CPUtime=0.08 cores=1,3,5,7
/proc/28880/stat : 28880 (python) S 28874 28874 28027 0 -1 4202560 902 0 0 0 4 4 0 0 18 0 1 0 670737236 286593024 5904 33554432000 4194304 5683644 140734205892368 18446744073709551615 248089718287 0 0 16781312 514 15279112646738018567 0 0 17 3 0 0 0
/proc/28880/statm: 69969 5904 377 364 0 5887 0
[pid=28882] ppid=28880 vsize=1018076 CPUtime=24.65 cores=1,3,5,7
/proc/28882/stat : 28882 (gnovelty+2) R 28880 28882 28882 0 -1 4202496 531 0 0 0 2465 0 0 0 25 0 1 0 670737236 1042509824 267 33554432000 134512640 135663704 4291542800 18446744073709551615 134525255 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/28882/statm: 254519 267 113 282 0 254235 0
[pid=28874/tid=28881] ppid=28872 vsize=290120 CPUtime=0 cores=1,3,5,7
/proc/28874/task/28881/stat : 28881 (python) S 28872 28874 28027 0 -1 4202560 31 933 0 0 0 0 0 0 20 0 2 0 670737236 297082880 7408 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 2 18446604457759544576 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 25.47
Current children cumulated vsize (KiB) 1588072

[startup+51.1006 s]
/proc/loadavg: 4.03 4.07 4.02 5/462 28883
/proc/meminfo: memFree=13958184/32950928 swapFree=62427640/67111528
[pid=28874] ppid=28872 vsize=290120 CPUtime=0.74 cores=1,3,5,7
/proc/28874/stat : 28874 (python) S 28872 28874 28027 0 -1 4202496 10089 933 1 0 67 7 0 0 20 0 2 0 670737152 297082880 7408 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/28874/statm: 72530 7408 1877 364 0 8448 0
[pid=28880] ppid=28874 vsize=279876 CPUtime=0.15 cores=1,3,5,7
/proc/28880/stat : 28880 (python) S 28874 28874 28027 0 -1 4202560 913 0 0 0 7 8 0 0 18 0 1 0 670737236 286593024 5904 33554432000 4194304 5683644 140734205892368 18446744073709551615 248089718287 0 0 16781312 514 15279112646738018567 0 0 17 7 0 0 0
/proc/28880/statm: 69969 5904 377 364 0 5887 0
[pid=28882] ppid=28880 vsize=1018076 CPUtime=50.25 cores=1,3,5,7
/proc/28882/stat : 28882 (gnovelty+2) R 28880 28882 28882 0 -1 4202496 531 0 0 0 5025 0 0 0 25 0 1 0 670737236 1042509824 267 33554432000 134512640 135663704 4291542800 18446744073709551615 134525276 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/28882/statm: 254519 267 113 282 0 254235 0
[pid=28874/tid=28881] ppid=28872 vsize=290120 CPUtime=0 cores=1,3,5,7
/proc/28874/task/28881/stat : 28881 (python) S 28872 28874 28027 0 -1 4202560 31 933 0 0 0 0 0 0 20 0 2 0 670737236 297082880 7408 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 2 18446604457759544576 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 51.14
Current children cumulated vsize (KiB) 1588072

[startup+102.317 s]
/proc/loadavg: 4.01 4.06 4.01 5/466 28889
/proc/meminfo: memFree=13946208/32950928 swapFree=62427640/67111528
[pid=28874] ppid=28872 vsize=300376 CPUtime=0.78 cores=1,3,5,7
/proc/28874/stat : 28874 (python) S 28872 28874 28027 0 -1 4202496 10963 933 1 0 70 8 0 0 20 0 3 0 670737152 307585024 7415 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/28874/statm: 75094 7415 1880 364 0 11009 0
[pid=28880] ppid=28874 vsize=290120 CPUtime=0.33 cores=1,3,5,7
/proc/28880/stat : 28880 (python) S 28874 28874 28027 0 -1 4202560 1085 0 0 0 17 16 0 0 19 0 2 0 670737236 297082880 5942 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/28880/statm: 72530 5942 410 364 0 8448 0
[pid=28882] ppid=28880 vsize=1018076 CPUtime=98.34 cores=1,3,5,7
/proc/28882/stat : 28882 (gnovelty+2) T 28880 28882 28882 0 -1 4202496 531 0 0 0 9834 0 0 0 25 0 1 0 670737236 1042509824 267 33554432000 134512640 135663704 4291542800 18446744073709551615 134524553 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/28882/statm: 254519 267 113 282 0 254235 0
[pid=28880/tid=28886] ppid=28874 vsize=290120 CPUtime=0 cores=1,3,5,7
/proc/28880/task/28886/stat : 28886 (python) S 28874 28874 28027 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 670747071 297082880 5942 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 514 18446604462541184064 0 0 -1 3 0 0 0
[pid=28887] ppid=28874 vsize=290132 CPUtime=0 cores=1,3,5,7
/proc/28887/stat : 28887 (python) S 28874 28874 28027 0 -1 4202560 743 0 0 0 0 0 0 0 18 0 1 0 670747074 297095168 5879 33554432000 4194304 5683644 140734205892368 18446744073709551615 248089718342 0 0 16781312 514 16090592089628180740 0 0 17 5 0 0 0
/proc/28887/statm: 72533 5879 348 364 0 8448 0
[pid=28889] ppid=28887 vsize=1018076 CPUtime=3.08 cores=1,3,5,7
/proc/28889/stat : 28889 (gnovelty+2) R 28887 28889 28889 0 -1 4202496 532 0 0 0 308 0 0 0 25 0 1 0 670747074 1042509824 267 33554432000 134512640 135663704 4290610560 18446744073709551615 134524882 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/28889/statm: 254519 267 113 282 0 254235 0
[pid=28874/tid=28881] ppid=28872 vsize=300376 CPUtime=0 cores=1,3,5,7
/proc/28874/task/28881/stat : 28881 (python) S 28872 28874 28027 0 -1 4202560 31 933 0 0 0 0 0 0 20 0 3 0 670737236 307585024 7415 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 2 18446604457759544576 0 0 -1 7 0 0 0
[pid=28874/tid=28888] ppid=28872 vsize=300376 CPUtime=0 cores=1,3,5,7
/proc/28874/task/28888/stat : 28888 (python) S 28872 28874 28027 0 -1 4202560 26 933 0 0 0 0 0 0 20 0 3 0 670747074 307585024 7415 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 2 18446604457759544576 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 102.53
Current children cumulated vsize (KiB) 2916780

[startup+162.301 s]
/proc/loadavg: 4.00 4.04 4.01 5/466 28890
/proc/meminfo: memFree=13937228/32950928 swapFree=62427640/67111528
[pid=28874] ppid=28872 vsize=300376 CPUtime=0.78 cores=1,3,5,7
/proc/28874/stat : 28874 (python) S 28872 28874 28027 0 -1 4202496 10963 933 1 0 70 8 0 0 20 0 3 0 670737152 307585024 7415 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/28874/statm: 75094 7415 1880 364 0 11009 0
[pid=28880] ppid=28874 vsize=290120 CPUtime=0.33 cores=1,3,5,7

################
# More data... #
################

Current children cumulated CPU time (s) 5000.02
Current children cumulated vsize (KiB) 11815268

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+4982.1 s]
/proc/loadavg: 4.05 4.07 4.02 5/519 29215
/proc/meminfo: memFree=13615268/32950928 swapFree=62427640/67111528
[pid=28874] ppid=28872 vsize=423820 CPUtime=1.38 cores=1,3,5,7
/proc/28874/stat : 28874 (python) S 28872 28874 28027 0 -1 4202496 23639 933 1 0 126 12 0 0 18 0 15 0 670737152 433991680 7574 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/28874/statm: 105955 7574 1904 364 0 41834 0
[pid=28880] ppid=28874 vsize=290120 CPUtime=0.33 cores=1,3,5,7
/proc/28880/stat : 28880 (python) S 28874 28874 28027 0 -1 4202560 1085 0 0 0 17 16 0 0 19 0 2 0 670737236 297082880 5942 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/28880/statm: 72530 5942 410 364 0 8448 0
[pid=28882] ppid=28880 vsize=1018076 CPUtime=98.34 cores=1,3,5,7
/proc/28882/stat : 28882 (gnovelty+2) T 28880 28882 28882 0 -1 4202496 531 0 0 0 9834 0 0 0 25 0 1 0 670737236 1042509824 267 33554432000 134512640 135663704 4291542800 18446744073709551615 134524553 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/28882/statm: 254519 267 113 282 0 254235 0
[pid=28880/tid=28886] ppid=28874 vsize=290120 CPUtime=0 cores=1,3,5,7
/proc/28880/task/28886/stat : 28886 (python) S 28874 28874 28027 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 670747071 297082880 5942 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 514 18446604462541184064 0 0 -1 3 0 0 0
[pid=28887] ppid=28874 vsize=290132 CPUtime=0.33 cores=1,3,5,7
/proc/28887/stat : 28887 (python) S 28874 28874 28027 0 -1 4202560 1075 0 0 0 17 16 0 0 19 0 2 0 670747074 297095168 5932 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/28887/statm: 72533 5932 400 364 0 8448 0
[pid=28889] ppid=28887 vsize=1018076 CPUtime=98.34 cores=1,3,5,7
/proc/28889/stat : 28889 (gnovelty+2) T 28887 28889 28889 0 -1 4202496 532 0 0 0 9834 0 0 0 25 0 1 0 670747074 1042509824 267 33554432000 134512640 135663704 4290610560 18446744073709551615 134524882 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/28889/statm: 254519 267 113 282 0 254235 0
[pid=28887/tid=28892] ppid=28874 vsize=290132 CPUtime=0 cores=1,3,5,7
/proc/28887/task/28892/stat : 28892 (python) S 28874 28874 28027 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 670756909 297095168 5932 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 514 18446604456659076160 0 0 -1 7 0 0 0
[pid=28893] ppid=28874 vsize=300388 CPUtime=0.7 cores=1,3,5,7
/proc/28893/stat : 28893 (python) S 28874 28874 28027 0 -1 4202560 1133 0 0 0 36 34 0 0 18 0 2 0 670756912 307597312 5936 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/28893/statm: 75097 5936 400 364 0 11009 0
[pid=28895] ppid=28893 vsize=416920 CPUtime=195.66 cores=1,3,5,7
/proc/28895/stat : 28895 (adaptg2wsat2009) T 28893 28895 28895 0 -1 4202496 673 0 0 0 19566 0 0 0 25 0 1 0 670756913 426926080 406 33554432000 134512640 135107411 4293724960 18446744073709551615 134531830 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/28895/statm: 104230 406 59 146 0 103106 0
[pid=28893/tid=28900] ppid=28874 vsize=300388 CPUtime=0 cores=1,3,5,7
/proc/28893/task/28900/stat : 28900 (python) S 28874 28874 28027 0 -1 4202560 48 0 0 0 0 0 0 0 18 0 2 0 670766846 307597312 5936 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 514 18446604463450076288 0 0 -1 7 0 0 0
[pid=28901] ppid=28874 vsize=310644 CPUtime=0.33 cores=1,3,5,7
/proc/28901/stat : 28901 (python) S 28874 28874 28027 0 -1 4202560 1106 0 0 0 18 15 0 0 19 0 2 0 670766849 318099456 5940 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/28901/statm: 77661 5940 400 364 0 13570 0
[pid=28903] ppid=28901 vsize=416920 CPUtime=100.25 cores=1,3,5,7
/proc/28903/stat : 28903 (adaptg2wsat2009) T 28901 28903 28903 0 -1 4202496 674 0 0 0 10025 0 0 0 25 0 1 0 670766849 426926080 407 33554432000 134512640 135107411 4291388432 18446744073709551615 134531656 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/28903/statm: 104230 407 59 146 0 103106 0
[pid=28901/tid=28909] ppid=28874 vsize=310644 CPUtime=0 cores=1,3,5,7
/proc/28901/task/28909/stat : 28909 (python) S 28874 28874 28027 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 670776882 318099456 5940 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 514 18446604466280109056 0 0 -1 7 0 0 0
[pid=28910] ppid=28874 vsize=320900 CPUtime=0.34 cores=1,3,5,7
/proc/28910/stat : 28910 (python) S 28874 28874 28027 0 -1 4202560 1113 0 0 0 19 15 0 0 19 0 2 0 670776885 328601600 5944 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/28910/statm: 80225 5944 400 364 0 16131 0
[pid=28912] ppid=28910 vsize=416920 CPUtime=99.33 cores=1,3,5,7
/proc/28912/stat : 28912 (adaptg2wsat2009) T 28910 28912 28912 0 -1 4202496 674 0 0 0 9933 0 0 0 25 0 1 0 670776885 426926080 407 33554432000 134512640 135107411 4294132384 18446744073709551615 134531822 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/28912/statm: 104230 407 59 146 0 103106 0
[pid=28910/tid=28916] ppid=28874 vsize=320900 CPUtime=0 cores=1,3,5,7
/proc/28910/task/28916/stat : 28916 (python) S 28874 28874 28027 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 670786819 328601600 5944 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 514 18446604470834717120 0 0 -1 7 0 0 0
[pid=28917] ppid=28874 vsize=331156 CPUtime=0.34 cores=1,3,5,7
/proc/28917/stat : 28917 (python) S 28874 28874 28027 0 -1 4202560 1120 0 0 0 18 16 0 0 19 0 2 0 670786823 339103744 5948 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/28917/statm: 82789 5948 400 364 0 18692 0
[pid=28919] ppid=28917 vsize=1018076 CPUtime=100.33 cores=1,3,5,7
/proc/28919/stat : 28919 (gnovelty+2) T 28917 28919 28919 0 -1 4202496 532 0 0 0 10033 0 0 0 25 0 1 0 670786823 1042509824 266 33554432000 134512640 135663704 4290310624 18446744073709551615 134525678 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/28919/statm: 254519 266 113 282 0 254235 0
[pid=28917/tid=28923] ppid=28874 vsize=331156 CPUtime=0 cores=1,3,5,7
/proc/28917/task/28923/stat : 28923 (python) S 28874 28874 28027 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 670796857 339103744 5948 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 514 18446604462803011648 0 0 -1 7 0 0 0
[pid=28924] ppid=28874 vsize=341412 CPUtime=0.35 cores=1,3,5,7
/proc/28924/stat : 28924 (python) S 28874 28874 28027 0 -1 4202560 1120 0 0 0 18 17 0 0 19 0 2 0 670796861 349605888 5952 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/28924/statm: 85353 5952 400 364 0 21253 0
[pid=28926] ppid=28924 vsize=247064 CPUtime=99.34 cores=1,3,5,7
/proc/28926/stat : 28926 (FH) T 28924 28926 28926 0 -1 4202496 632 0 0 0 9934 0 0 0 25 0 1 0 670796861 252993536 365 33554432000 134512640 134936988 4292178256 18446744073709551615 134554216 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/28926/statm: 61766 365 41 104 0 61270 0
[pid=28924/tid=28929] ppid=28874 vsize=341412 CPUtime=0 cores=1,3,5,7
/proc/28924/task/28929/stat : 28929 (python) S 28874 28874 28027 0 -1 4202560 51 0 0 0 0 0 0 0 19 0 2 0 670806796 349605888 5952 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 514 18446604455982418112 0 0 -1 7 0 0 0
[pid=28930] ppid=28874 vsize=351668 CPUtime=8.45 cores=1,3,5,7
/proc/28930/stat : 28930 (python) S 28874 28874 28027 0 -1 4202560 1209 0 0 0 439 406 0 0 15 0 2 0 670806799 360108032 5962 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/28930/statm: 87917 5962 406 364 0 23814 0
[pid=28932] ppid=28930 vsize=14496 CPUtime=2233.44 cores=1,3,5,7
/proc/28932/stat : 28932 (march_hi) T 28930 28932 28932 0 -1 4202496 3907 0 0 0 223343 1 0 0 25 0 1 0 670806800 14843904 2869 33554432000 4194304 4955518 140736995115456 18446744073709551615 4286020 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/28932/statm: 3624 2869 106 186 0 3437 0
[pid=28930/tid=28948] ppid=28874 vsize=351668 CPUtime=0 cores=1,3,5,7
/proc/28930/task/28948/stat : 28948 (python) S 28874 28874 28027 0 -1 4202560 49 0 0 0 0 0 0 0 15 0 2 0 670845854 360108032 5962 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 514 18446604464427337728 0 0 -1 3 0 0 0
[pid=28960] ppid=28874 vsize=361924 CPUtime=0.35 cores=1,3,5,7
/proc/28960/stat : 28960 (python) S 28874 28874 28027 0 -1 4202560 1139 0 0 0 18 17 0 0 17 0 2 0 670885006 370610176 5960 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/28960/statm: 90481 5960 400 364 0 26375 0
[pid=28962] ppid=28960 vsize=416920 CPUtime=100.26 cores=1,3,5,7
/proc/28962/stat : 28962 (adaptg2wsat2009) T 28960 28962 28962 0 -1 4202496 682 0 0 0 10026 0 0 0 25 0 1 0 670885007 426926080 406 33554432000 134512640 135107411 4292844016 18446744073709551615 134532121 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/28962/statm: 104230 406 59 146 0 103106 0
[pid=28960/tid=28969] ppid=28874 vsize=361924 CPUtime=0 cores=1,3,5,7
/proc/28960/task/28969/stat : 28969 (python) S 28874 28874 28027 0 -1 4202560 51 0 0 0 0 0 0 0 17 0 2 0 670895042 370610176 5960 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 514 18446604464427336832 0 0 -1 7 0 0 0
[pid=28970] ppid=28874 vsize=372180 CPUtime=2.94 cores=1,3,5,7
/proc/28970/stat : 28970 (python) S 28874 28874 28027 0 -1 4202560 1179 0 0 0 149 145 0 0 15 0 2 0 670895045 381112320 5964 33554432000 4194304 5683644 140734205892368 18446744073709551615 248089718342 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/28970/statm: 93045 5964 400 364 0 28936 0
[pid=28972] ppid=28970 vsize=6304 CPUtime=776.7 cores=1,3,5,7
/proc/28972/stat : 28972 (march_hi) R 28970 28972 28972 0 -1 4202496 2290 0 0 0 77670 0 0 0 25 0 1 0 670895045 6455296 1252 33554432000 4194304 4955518 140733433156784 18446744073709551615 4208784 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/28972/statm: 1576 1252 106 186 0 1389 0
[pid=28970/tid=28983] ppid=28874 vsize=372180 CPUtime=0 cores=1,3,5,7
/proc/28970/task/28983/stat : 28983 (python) S 28874 28874 28027 0 -1 4202560 47 0 0 0 0 0 0 0 15 0 2 0 670934287 381112320 5964 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 514 18446604470834718016 0 0 -1 7 0 0 0
[pid=28997] ppid=28874 vsize=382436 CPUtime=2.62 cores=1,3,5,7
/proc/28997/stat : 28997 (python) S 28874 28874 28027 0 -1 4202560 1176 0 0 0 143 119 0 0 15 0 2 0 670963510 391614464 5968 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/28997/statm: 95609 5968 400 364 0 31497 0
[pid=28999] ppid=28997 vsize=6304 CPUtime=682.82 cores=1,3,5,7
/proc/28999/stat : 28999 (march_hi) T 28997 28999 28999 0 -1 4202496 2192 0 0 0 68282 0 0 0 25 0 1 0 670963510 6455296 1157 33554432000 4194304 4955518 140737117028208 18446744073709551615 4297477 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/28999/statm: 1576 1157 106 186 0 1389 0
[pid=28997/tid=29011] ppid=28874 vsize=382436 CPUtime=0 cores=1,3,5,7
/proc/28997/task/29011/stat : 29011 (python) S 28874 28874 28027 0 -1 4202560 49 0 0 0 0 0 0 0 15 0 2 0 671002459 391614464 5968 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 514 18446604455231493120 0 0 -1 7 0 0 0
[pid=29034] ppid=28874 vsize=392868 CPUtime=0.35 cores=1,3,5,7
/proc/29034/stat : 29034 (python) S 28874 28874 28027 0 -1 4202560 1137 0 0 0 18 17 0 0 19 0 2 0 671041440 402296832 6010 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/29034/statm: 98217 6010 400 364 0 34102 0
[pid=29036] ppid=29034 vsize=247068 CPUtime=99.34 cores=1,3,5,7
/proc/29036/stat : 29036 (FH) T 29034 29036 29036 0 -1 4202496 638 0 0 0 9934 0 0 0 25 0 1 0 671041440 252997632 365 33554432000 134512640 134936988 4291472080 18446744073709551615 134554071 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/29036/statm: 61767 365 41 104 0 61271 0
[pid=29034/tid=29040] ppid=28874 vsize=392868 CPUtime=0 cores=1,3,5,7
/proc/29034/task/29040/stat : 29040 (python) S 28874 28874 28027 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 671051376 402296832 6010 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 514 18446604468044408832 0 0 -1 7 0 0 0
[pid=29041] ppid=28874 vsize=403124 CPUtime=0.7 cores=1,3,5,7
/proc/29041/stat : 29041 (python) S 28874 28874 28027 0 -1 4202560 1156 0 0 0 38 32 0 0 19 0 2 0 671051378 412798976 6014 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/29041/statm: 100781 6014 400 364 0 36663 0
[pid=29043] ppid=29041 vsize=268420 CPUtime=195.73 cores=1,3,5,7
/proc/29043/stat : 29043 (hybridGM3) T 29041 29043 29043 0 -1 4202496 4897 0 0 0 19573 0 0 0 25 0 1 0 671051379 274862080 4628 33554432000 134512640 135204385 4286882576 18446744073709551615 134536744 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/29043/statm: 67105 4628 64 169 0 66935 0
[pid=29041/tid=29049] ppid=28874 vsize=403124 CPUtime=0 cores=1,3,5,7
/proc/29041/task/29049/stat : 29049 (python) S 28874 28874 28027 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 671070952 412798976 6014 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 514 18446604460303743744 0 0 -1 7 0 0 0
[pid=29050] ppid=28874 vsize=412856 CPUtime=0.36 cores=1,3,5,7
/proc/29050/stat : 29050 (python) S 28874 28874 28027 0 -1 4202560 1155 0 0 0 18 18 0 0 19 0 2 0 671070954 422764544 5888 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/29050/statm: 103214 5888 400 364 0 39093 0
[pid=29052] ppid=29050 vsize=1018076 CPUtime=100.27 cores=1,3,5,7
/proc/29052/stat : 29052 (gnovelty+2) T 29050 29052 29052 0 -1 4202496 540 0 0 0 10027 0 0 0 25 0 1 0 671070955 1042509824 267 33554432000 134512640 135663704 4289407088 18446744073709551615 134525070 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/29052/statm: 254519 267 113 282 0 254235 0
[pid=29050/tid=29056] ppid=28874 vsize=412856 CPUtime=0 cores=1,3,5,7
/proc/29050/task/29056/stat : 29056 (python) S 28874 28874 28027 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 671080993 422764544 5888 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 514 18446604461621039424 0 0 -1 7 0 0 0
[pid=28874/tid=28881] ppid=28872 vsize=423820 CPUtime=0 cores=1,3,5,7
/proc/28874/task/28881/stat : 28881 (python) S 28872 28874 28027 0 -1 4202560 31 933 0 0 0 0 0 0 20 0 15 0 670737236 433991680 7574 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 2 18446604457759544576 0 0 -1 7 0 0 0
[pid=28874/tid=28888] ppid=28872 vsize=423820 CPUtime=0 cores=1,3,5,7
/proc/28874/task/28888/stat : 28888 (python) S 28872 28874 28027 0 -1 4202560 26 933 0 0 0 0 0 0 20 0 15 0 670747074 433991680 7574 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 2 18446604457759544576 0 0 -1 5 0 0 0
[pid=28874/tid=28894] ppid=28872 vsize=423820 CPUtime=0 cores=1,3,5,7
/proc/28874/task/28894/stat : 28894 (python) S 28872 28874 28027 0 -1 4202560 33 933 0 0 0 0 0 0 18 0 15 0 670756912 433991680 7574 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 2 18446604457759544576 0 0 -1 5 0 0 0
[pid=28874/tid=28902] ppid=28872 vsize=423820 CPUtime=0 cores=1,3,5,7
/proc/28874/task/28902/stat : 28902 (python) S 28872 28874 28027 0 -1 4202560 24 933 0 0 0 0 0 0 20 0 15 0 670766849 433991680 7574 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 2 18446604457759544576 0 0 -1 5 0 0 0
[pid=28874/tid=28911] ppid=28872 vsize=423820 CPUtime=0 cores=1,3,5,7
/proc/28874/task/28911/stat : 28911 (python) S 28872 28874 28027 0 -1 4202560 31 933 0 0 0 0 0 0 20 0 15 0 670776885 433991680 7574 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 2 18446604457759544576 0 0 -1 5 0 0 0
[pid=28874/tid=28918] ppid=28872 vsize=423820 CPUtime=0 cores=1,3,5,7
/proc/28874/task/28918/stat : 28918 (python) S 28872 28874 28027 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 15 0 670786823 433991680 7574 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 2 18446604457759544576 0 0 -1 3 0 0 0
[pid=28874/tid=28925] ppid=28872 vsize=423820 CPUtime=0 cores=1,3,5,7
/proc/28874/task/28925/stat : 28925 (python) S 28872 28874 28027 0 -1 4202560 31 933 0 0 0 0 0 0 20 0 15 0 670796861 433991680 7574 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 2 18446604457759544576 0 0 -1 3 0 0 0
[pid=28874/tid=28931] ppid=28872 vsize=423820 CPUtime=0 cores=1,3,5,7
/proc/28874/task/28931/stat : 28931 (python) S 28872 28874 28027 0 -1 4202560 31 933 0 0 0 0 0 0 18 0 15 0 670806799 433991680 7574 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 2 18446604457759544576 0 0 -1 3 0 0 0
[pid=28874/tid=28961] ppid=28872 vsize=423820 CPUtime=0 cores=1,3,5,7
/proc/28874/task/28961/stat : 28961 (python) S 28872 28874 28027 0 -1 4202560 29 933 0 0 0 0 0 0 20 0 15 0 670885007 433991680 7574 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 2 18446604457759544576 0 0 -1 5 0 0 0
[pid=28874/tid=28971] ppid=28872 vsize=423820 CPUtime=0 cores=1,3,5,7
/proc/28874/task/28971/stat : 28971 (python) S 28872 28874 28027 0 -1 4202560 34 933 0 0 0 0 0 0 18 0 15 0 670895045 433991680 7574 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 2 18446604457759544576 0 0 -1 7 0 0 0
[pid=28874/tid=28998] ppid=28872 vsize=423820 CPUtime=0 cores=1,3,5,7
/proc/28874/task/28998/stat : 28998 (python) S 28872 28874 28027 0 -1 4202560 28 933 0 0 0 0 0 0 18 0 15 0 670963510 433991680 7574 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 2 18446604457759544576 0 0 -1 5 0 0 0
[pid=28874/tid=29035] ppid=28872 vsize=423820 CPUtime=0 cores=1,3,5,7
/proc/28874/task/29035/stat : 29035 (python) S 28872 28874 28027 0 -1 4202560 26 933 0 0 0 0 0 0 20 0 15 0 671041440 433991680 7574 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 2 18446604457759544576 0 0 -1 5 0 0 0
[pid=28874/tid=29042] ppid=28872 vsize=423820 CPUtime=0 cores=1,3,5,7
/proc/28874/task/29042/stat : 29042 (python) S 28872 28874 28027 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 15 0 671051378 433991680 7574 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 2 18446604457759544576 0 0 -1 3 0 0 0
[pid=28874/tid=29051] ppid=28872 vsize=423820 CPUtime=0 cores=1,3,5,7
/proc/28874/task/29051/stat : 29051 (python) S 28872 28874 28027 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 15 0 671070955 433991680 7574 33554432000 4194304 5683644 140734205892368 18446744073709551615 248101521841 0 0 16781312 2 18446604457759544576 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 5000.02
Current children cumulated vsize (KiB) 11815268

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 28874 and gives
#  childrusage.ru_utime.tv_sec=1865
#  childrusage.ru_utime.tv_usec=985327
#  childrusage.ru_stime.tv_sec=9
#  childrusage.ru_stime.tv_usec=66621
# CPU time returned by wait4() is 1875.05
# while last known CPU time is 5000.02
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 4982.2
CPU time (s): 5000.02
CPU user time (s): 4991.06
CPU system time (s): 8.96
CPU usage (%): 100.358
Max. virtual memory (cumulated for all children) (KiB): 11815272

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1865.99
system time used= 9.06662
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 54702
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 7512
involuntary context switches= 12933

runsolver used 35.6046 second user time and 82.3985 second system time

The end

Launcher Data 

Begin job on node123 at 2011-05-17 02:16:13
IDJOB=3386752
IDBENCH=82817
IDSOLVER=1872
FILE ID=node123/3386752-1305591373
RUNJOBID= node123-1305586147-28047
PBS_JOBID= 13350355
Free space on /tmp= 73200 MiB

SOLVER NAME= borg-sat 11.03.02-seq-random (fixed)
BENCH NAME= SAT11/random/medium/unif-k7-r89-v70-c6230-S1475900283-041.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3386752-1305591373/watcher-3386752-1305591373 -o /tmp/evaluation-result-3386752-1305591373/solver-3386752-1305591373 -C 5000 -W 5100 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3386752-1305591373.cnf -seed 779752522 -budget 5000

TIME LIMIT= 5000 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= a83cf27d1d664cde32ad2f1a49b00933
RANDOM SEED=779752522

node123.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      13990680 kB
Buffers:         15780 kB
Cached:         230964 kB
SwapCached:      75220 kB
Active:        6501300 kB
Inactive:     12219532 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      13990680 kB
SwapTotal:    67111528 kB
SwapFree:     62427640 kB
Dirty:           32212 kB
Writeback:           0 kB
AnonPages:    18406608 kB
Mapped:          34180 kB
Slab:            91628 kB
PageTables:      79296 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 126303660 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73164 MiB
End job on node123 at 2011-05-17 03:39:18