Trace number 3386675

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-seq-random (fixed)? (TO) 5000.11 4978.4

General information on the benchmark

Namerandom/medium/
unif-k7-r89-v70-c6230-S1254276596-021.UNKNOWN.cnf
MD5SUM38eb341db913fde0f11427c4d808f2c9
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark4367.31
Satisfiable
(Un)Satisfiability was proved
Number of variables70
Number of clauses6230
Sum of the clauses size43610
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 56230

Solver Data 

0.40/0.67	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.40/0.67	c INFO: solving HOME/instance-3386675-1305590341.cnf
0.40/0.67	c INFO: computing task features
0.47/0.71	c INFO: computed variable-clause graph statistics
0.47/0.72	c INFO: computed clause balance statistics
0.67/0.94	c INFO: computed variable balance statistics
0.79/1.03	c INFO: computed small-clause counts
0.79/1.03	c INFO: computed Horn-clause counts
0.79/1.04	c INFO: computed variable graph statistics
0.79/1.04	c INFO: collected features for HOME/instance-3386675-1305590341.cnf in 0.37 s
0.79/1.07	c INFO: running gnovelty+2@0 for 97 with 4999 remaining (b = 0.53)
100.65/100.54	c INFO: running gnovelty+2@0 for 97 with 4900 remaining (b = 0.10)
201.32/200.98	c INFO: running adaptg2wsat2009++@0 for 97 with 4799 remaining (b = 0.05)
302.08/301.41	c INFO: running adaptg2wsat2009++@0 for 97 with 4699 remaining (b = 0.04)
401.98/400.90	c INFO: running adaptg2wsat2009++@0 for 97 with 4600 remaining (b = 0.03)
502.89/501.35	c INFO: running gnovelty+2@0 for 97 with 4499 remaining (b = 0.03)
600.80/598.85	c INFO: running FH@0 for 97 with 4402 remaining (b = 0.02)
700.74/698.33	c INFO: running march_hi@0 for 388 with 4302 remaining (b = 0.08)
1094.31/1090.17	c INFO: running march_hi@391 for 97 with 3910 remaining (b = 0.03)
1193.15/1188.53	c INFO: running march_hi@490 for 194 with 3812 remaining (b = 0.05)
1384.83/1379.36	c INFO: running march_hi@680 for 97 with 3621 remaining (b = 0.02)
1481.78/1475.81	c INFO: running adaptg2wsat2009++@0 for 97 with 3525 remaining (b = 0.02)
1582.60/1576.27	c INFO: running march_hi@0 for 388 with 3424 remaining (b = 0.07)
1973.81/1965.67	c INFO: running march_hi@389 for 97 with 3035 remaining (b = 0.03)
2070.67/2062.04	c INFO: running march_hi@485 for 194 with 2938 remaining (b = 0.04)
2269.76/2260.27	c INFO: running march_hi@0 for 388 with 2740 remaining (b = 0.06)
2661.87/2650.57	c INFO: running march_hi@390 for 97 with 2350 remaining (b = 0.02)
2758.23/2746.44	c INFO: running march_hi@486 for 194 with 2254 remaining (b = 0.03)
2954.61/2941.90	c INFO: running FH@0 for 97 with 2059 remaining (b = 0.01)
3054.38/3041.36	c INFO: running adaptg2wsat2009++@100 for 97 with 1959 remaining (b = 0.01)
3154.35/3140.84	c INFO: running gnovelty+2@0 for 97 with 1860 remaining (b = 0.01)
3253.27/3239.31	c INFO: running hybridGM3@0 for 194 with 1761 remaining (b = 0.02)
3452.02/3437.21	c INFO: running march_hi@777 for 1358 with 1563 remaining (b = 0.17)
4816.85/4795.99	c INFO: running march_hi@2136 for 97 with 205 remaining (b = 0.04)
4915.79/4894.42	c INFO: running march_hi@683 for 106 with 106 remaining (b = 0.01)
5000.11/4978.38	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3386675-1305590341/watcher-3386675-1305590341 -o /tmp/evaluation-result-3386675-1305590341/solver-3386675-1305590341 -C 5000 -W 5100 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3386675-1305590341.cnf -seed 1072918822 -budget 5000 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 5030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.00 2.01 2.00 4/561 21814
/proc/meminfo: memFree=27717804/32950928 swapFree=41934280/67111528
[pid=21814] ppid=21812 vsize=16236 CPUtime=0 cores=0,2,4,6
/proc/21814/stat : 21814 (python) R 21812 21814 21157 0 -1 4202496 1043 0 0 0 0 0 0 0 25 0 1 0 670632849 16625664 916 33554432000 4194304 5683644 140736734403616 18446744073709551615 251813934036 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/21814/statm: 4059 916 381 364 0 546 0

[startup+0.0551781 s]
/proc/loadavg: 2.00 2.01 2.00 4/561 21814
/proc/meminfo: memFree=27717804/32950928 swapFree=41934280/67111528
[pid=21814] ppid=21812 vsize=25292 CPUtime=0.04 cores=0,2,4,6
/proc/21814/stat : 21814 (python) S 21812 21814 21157 0 -1 4202496 1346 0 0 0 3 1 0 0 25 0 1 0 670632849 25899008 1193 33554432000 4194304 5683644 140736734403616 18446744073709551615 251809316487 0 2147483391 16781312 2 18446744071699215550 0 0 17 2 0 0 0
/proc/21814/statm: 6323 1193 438 364 0 724 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 25292

[startup+0.10117 s]
/proc/loadavg: 2.00 2.01 2.00 4/561 21814
/proc/meminfo: memFree=27717804/32950928 swapFree=41934280/67111528
[pid=21814] ppid=21812 vsize=87704 CPUtime=0.07 cores=0,2,4,6
/proc/21814/stat : 21814 (python) S 21812 21814 21157 0 -1 4202496 2636 0 0 0 5 2 0 0 18 0 1 0 670632849 89808896 2354 33554432000 4194304 5683644 140736734403616 18446744073709551615 251814238208 0 2147483391 16781312 2 18446744071699215550 0 0 17 2 0 0 0
/proc/21814/statm: 21926 2354 717 364 0 1622 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 87704

[startup+0.300123 s]
/proc/loadavg: 2.00 2.01 2.00 4/561 21814
/proc/meminfo: memFree=27717804/32950928 swapFree=41934280/67111528
[pid=21814] ppid=21812 vsize=135748 CPUtime=0.21 cores=0,2,4,6
/proc/21814/stat : 21814 (python) S 21812 21814 21157 0 -1 4202496 5972 931 1 0 15 6 0 0 18 0 1 0 670632849 139005952 4149 33554432000 4194304 5683644 140736734403616 18446744073709551615 251814236165 0 2147483391 16781312 2 18446744071699215550 0 0 17 2 0 0 0
/proc/21814/statm: 33937 4149 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.21
Current children cumulated vsize (KiB) 135748

[startup+0.701057 s]
/proc/loadavg: 2.00 2.01 2.00 4/561 21814
/proc/meminfo: memFree=27717804/32950928 swapFree=41934280/67111528
[pid=21814] ppid=21812 vsize=279516 CPUtime=0.47 cores=0,2,4,6
/proc/21814/stat : 21814 (python) R 21812 21814 21157 0 -1 4202496 9530 931 1 0 34 13 0 0 17 0 1 0 670632849 286224384 7203 33554432000 4194304 5683644 140736734403616 18446744073709551615 4602560 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/21814/statm: 69879 7203 1761 364 0 5803 0
Current children cumulated CPU time (s) 0.47
Current children cumulated vsize (KiB) 279516

[startup+1.50087 s]
/proc/loadavg: 2.00 2.01 2.00 3/562 21819
/proc/meminfo: memFree=27693368/32950928 swapFree=41934280/67111528
[pid=21814] ppid=21812 vsize=290116 CPUtime=0.84 cores=0,2,4,6
/proc/21814/stat : 21814 (python) S 21812 21814 21157 0 -1 4202496 10086 931 1 0 71 13 0 0 20 0 2 0 670632849 297078784 7408 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/21814/statm: 72529 7408 1877 364 0 8447 0
Current children cumulated CPU time (s) 0.84
Current children cumulated vsize (KiB) 290116

[startup+3.10071 s]
/proc/loadavg: 2.00 2.01 2.00 3/565 21822
/proc/meminfo: memFree=27690124/32950928 swapFree=41934280/67111528
[pid=21814] ppid=21812 vsize=290116 CPUtime=0.84 cores=0,2,4,6
/proc/21814/stat : 21814 (python) S 21812 21814 21157 0 -1 4202496 10086 931 1 0 71 13 0 0 20 0 2 0 670632849 297078784 7408 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/21814/statm: 72529 7408 1877 364 0 8447 0
[pid=21820] ppid=21814 vsize=279872 CPUtime=0 cores=0,2,4,6
/proc/21820/stat : 21820 (python) S 21814 21814 21157 0 -1 4202560 746 0 0 0 0 0 0 0 18 0 1 0 670632957 286588928 5881 33554432000 4194304 5683644 140736734403616 18446744073709551615 251814260239 0 0 16781312 514 308553478496551171 0 0 17 2 0 0 0
/proc/21820/statm: 69968 5881 354 364 0 5886 0
[pid=21822] ppid=21820 vsize=1018076 CPUtime=2.02 cores=0,2,4,6
/proc/21822/stat : 21822 (gnovelty+2) R 21820 21822 21822 0 -1 4202496 530 0 0 0 202 0 0 0 25 0 1 0 670632957 1042509824 267 33554432000 134512640 135663704 4290551664 18446744073709551615 134525276 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/21822/statm: 254519 267 113 282 0 254235 0
[pid=21814/tid=21821] ppid=21812 vsize=290116 CPUtime=0 cores=0,2,4,6
/proc/21814/task/21821/stat : 21821 (python) S 21812 21814 21157 0 -1 4202560 32 931 0 0 0 0 0 0 20 0 2 0 670632957 297078784 7408 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 2 18446604452299776960 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 2.86
Current children cumulated vsize (KiB) 1588064

[startup+6.30118 s]
/proc/loadavg: 2.00 2.01 2.00 3/565 21823
/proc/meminfo: memFree=27689144/32950928 swapFree=41934280/67111528
[pid=21814] ppid=21812 vsize=290116 CPUtime=0.84 cores=0,2,4,6
/proc/21814/stat : 21814 (python) S 21812 21814 21157 0 -1 4202496 10086 931 1 0 71 13 0 0 20 0 2 0 670632849 297078784 7408 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/21814/statm: 72529 7408 1877 364 0 8447 0
[pid=21820] ppid=21814 vsize=279872 CPUtime=0.02 cores=0,2,4,6
/proc/21820/stat : 21820 (python) S 21814 21814 21157 0 -1 4202560 899 0 0 0 1 1 0 0 18 0 1 0 670632957 286588928 5904 33554432000 4194304 5683644 140736734403616 18446744073709551615 251814260239 0 0 16781312 514 308553478496551171 0 0 17 2 0 0 0
/proc/21820/statm: 69968 5904 377 364 0 5886 0
[pid=21822] ppid=21820 vsize=1018076 CPUtime=5.22 cores=0,2,4,6
/proc/21822/stat : 21822 (gnovelty+2) R 21820 21822 21822 0 -1 4202496 530 0 0 0 522 0 0 0 25 0 1 0 670632957 1042509824 267 33554432000 134512640 135663704 4290551664 18446744073709551615 134545635 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/21822/statm: 254519 267 113 282 0 254235 0
[pid=21814/tid=21821] ppid=21812 vsize=290116 CPUtime=0 cores=0,2,4,6
/proc/21814/task/21821/stat : 21821 (python) S 21812 21814 21157 0 -1 4202560 32 931 0 0 0 0 0 0 20 0 2 0 670632957 297078784 7408 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 2 18446604452299776960 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 6.08
Current children cumulated vsize (KiB) 1588064

[startup+12.7011 s]
/proc/loadavg: 2.16 2.05 2.01 3/565 21823
/proc/meminfo: memFree=27688948/32950928 swapFree=41934280/67111528
[pid=21814] ppid=21812 vsize=290116 CPUtime=0.84 cores=0,2,4,6
/proc/21814/stat : 21814 (python) S 21812 21814 21157 0 -1 4202496 10086 931 1 0 71 13 0 0 20 0 2 0 670632849 297078784 7408 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/21814/statm: 72529 7408 1877 364 0 8447 0
[pid=21820] ppid=21814 vsize=279872 CPUtime=0.04 cores=0,2,4,6
/proc/21820/stat : 21820 (python) S 21814 21814 21157 0 -1 4202560 902 0 0 0 2 2 0 0 18 0 1 0 670632957 286588928 5904 33554432000 4194304 5683644 140736734403616 18446744073709551615 251814260239 0 0 16781312 514 11760833767529218307 0 0 17 6 0 0 0
/proc/21820/statm: 69968 5904 377 364 0 5886 0
[pid=21822] ppid=21820 vsize=1018076 CPUtime=11.62 cores=0,2,4,6
/proc/21822/stat : 21822 (gnovelty+2) R 21820 21822 21822 0 -1 4202496 530 0 0 0 1162 0 0 0 25 0 1 0 670632957 1042509824 267 33554432000 134512640 135663704 4290551664 18446744073709551615 134525309 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/21822/statm: 254519 267 113 282 0 254235 0
[pid=21814/tid=21821] ppid=21812 vsize=290116 CPUtime=0 cores=0,2,4,6
/proc/21814/task/21821/stat : 21821 (python) S 21812 21814 21157 0 -1 4202560 32 931 0 0 0 0 0 0 20 0 2 0 670632957 297078784 7408 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 2 18446604452299776960 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 12.5
Current children cumulated vsize (KiB) 1588064

[startup+25.501 s]
/proc/loadavg: 2.12 2.04 2.01 3/565 21823
/proc/meminfo: memFree=27689284/32950928 swapFree=41934280/67111528
[pid=21814] ppid=21812 vsize=290116 CPUtime=0.84 cores=0,2,4,6
/proc/21814/stat : 21814 (python) S 21812 21814 21157 0 -1 4202496 10086 931 1 0 71 13 0 0 20 0 2 0 670632849 297078784 7408 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/21814/statm: 72529 7408 1877 364 0 8447 0
[pid=21820] ppid=21814 vsize=279872 CPUtime=0.09 cores=0,2,4,6
/proc/21820/stat : 21820 (python) S 21814 21814 21157 0 -1 4202560 908 0 0 0 5 4 0 0 18 0 1 0 670632957 286588928 5904 33554432000 4194304 5683644 140736734403616 18446744073709551615 251814260239 0 0 16781312 514 11760833767529218307 0 0 17 6 0 0 0
/proc/21820/statm: 69968 5904 377 364 0 5886 0
[pid=21822] ppid=21820 vsize=1018076 CPUtime=24.42 cores=0,2,4,6
/proc/21822/stat : 21822 (gnovelty+2) R 21820 21822 21822 0 -1 4202496 530 0 0 0 2442 0 0 0 25 0 1 0 670632957 1042509824 267 33554432000 134512640 135663704 4290551664 18446744073709551615 134524546 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/21822/statm: 254519 267 113 282 0 254235 0
[pid=21814/tid=21821] ppid=21812 vsize=290116 CPUtime=0 cores=0,2,4,6
/proc/21814/task/21821/stat : 21821 (python) S 21812 21814 21157 0 -1 4202560 32 931 0 0 0 0 0 0 20 0 2 0 670632957 297078784 7408 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 2 18446604452299776960 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 25.35
Current children cumulated vsize (KiB) 1588064

[startup+51.1007 s]
/proc/loadavg: 2.08 2.04 2.01 3/569 21828
/proc/meminfo: memFree=27676796/32950928 swapFree=41934280/67111528
[pid=21814] ppid=21812 vsize=290116 CPUtime=0.84 cores=0,2,4,6
/proc/21814/stat : 21814 (python) S 21812 21814 21157 0 -1 4202496 10086 931 1 0 71 13 0 0 20 0 2 0 670632849 297078784 7408 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/21814/statm: 72529 7408 1877 364 0 8447 0
[pid=21820] ppid=21814 vsize=279872 CPUtime=0.18 cores=0,2,4,6
/proc/21820/stat : 21820 (python) S 21814 21814 21157 0 -1 4202560 917 0 0 0 9 9 0 0 18 0 1 0 670632957 286588928 5904 33554432000 4194304 5683644 140736734403616 18446744073709551615 251814260239 0 0 16781312 514 11760833767529218307 0 0 17 6 0 0 0
/proc/21820/statm: 69968 5904 377 364 0 5886 0
[pid=21822] ppid=21820 vsize=1018076 CPUtime=50.02 cores=0,2,4,6
/proc/21822/stat : 21822 (gnovelty+2) R 21820 21822 21822 0 -1 4202496 530 0 0 0 5002 0 0 0 25 0 1 0 670632957 1042509824 267 33554432000 134512640 135663704 4290551664 18446744073709551615 134524882 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/21822/statm: 254519 267 113 282 0 254235 0
[pid=21814/tid=21821] ppid=21812 vsize=290116 CPUtime=0 cores=0,2,4,6
/proc/21814/task/21821/stat : 21821 (python) S 21812 21814 21157 0 -1 4202560 32 931 0 0 0 0 0 0 20 0 2 0 670632957 297078784 7408 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 2 18446604452299776960 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 51.04
Current children cumulated vsize (KiB) 1588064

[startup+102.32 s]
/proc/loadavg: 2.20 2.09 2.02 3/573 21833
/proc/meminfo: memFree=27671412/32950928 swapFree=41934280/67111528
[pid=21814] ppid=21812 vsize=300372 CPUtime=0.87 cores=0,2,4,6
/proc/21814/stat : 21814 (python) S 21812 21814 21157 0 -1 4202496 10982 931 1 0 74 13 0 0 19 0 3 0 670632849 307580928 7415 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/21814/statm: 75093 7415 1880 364 0 11008 0
[pid=21820] ppid=21814 vsize=290116 CPUtime=0.4 cores=0,2,4,6
/proc/21820/stat : 21820 (python) S 21814 21814 21157 0 -1 4202560 1092 0 0 0 21 19 0 0 19 0 2 0 670632957 297078784 5942 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/21820/statm: 72529 5942 410 364 0 8447 0
[pid=21822] ppid=21820 vsize=1018076 CPUtime=99.43 cores=0,2,4,6
/proc/21822/stat : 21822 (gnovelty+2) T 21820 21822 21822 0 -1 4202496 530 0 0 0 9943 0 0 0 25 0 1 0 670632957 1042509824 267 33554432000 134512640 135663704 4290551664 18446744073709551615 134525688 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/21822/statm: 254519 267 113 282 0 254235 0
[pid=21820/tid=21830] ppid=21814 vsize=290116 CPUtime=0 cores=0,2,4,6
/proc/21820/task/21830/stat : 21830 (python) S 21814 21814 21157 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 670642901 297078784 5942 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 514 18446604448689566656 0 0 -1 6 0 0 0
[pid=21831] ppid=21814 vsize=290128 CPUtime=0 cores=0,2,4,6
/proc/21831/stat : 21831 (python) S 21814 21814 21157 0 -1 4202560 747 0 0 0 0 0 0 0 18 0 1 0 670642904 297091072 5879 33554432000 4194304 5683644 140736734403616 18446744073709551615 251814260294 0 0 16781312 514 1104132604564570369 0 0 17 4 0 0 0
/proc/21831/statm: 72532 5879 348 364 0 8447 0
[pid=21833] ppid=21831 vsize=1018076 CPUtime=1.77 cores=0,2,4,6
/proc/21833/stat : 21833 (gnovelty+2) R 21831 21833 21833 0 -1 4202496 531 0 0 0 177 0 0 0 25 0 1 0 670642904 1042509824 268 33554432000 134512640 135663704 4286986000 18446744073709551615 134525286 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/21833/statm: 254519 268 113 282 0 254235 0
[pid=21814/tid=21821] ppid=21812 vsize=300372 CPUtime=0 cores=0,2,4,6
/proc/21814/task/21821/stat : 21821 (python) S 21812 21814 21157 0 -1 4202560 32 931 0 0 0 0 0 0 20 0 3 0 670632957 307580928 7415 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 2 18446604452299776960 0 0 -1 2 0 0 0
[pid=21814/tid=21832] ppid=21812 vsize=300372 CPUtime=0 cores=0,2,4,6
/proc/21814/task/21832/stat : 21832 (python) S 21812 21814 21157 0 -1 4202560 26 931 0 0 0 0 0 0 20 0 3 0 670642904 307580928 7415 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 2 18446604452299776960 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 102.47
Current children cumulated vsize (KiB) 2916768

[startup+162.301 s]
/proc/loadavg: 2.22 2.11 2.03 3/577 21848
/proc/meminfo: memFree=27656408/32950928 swapFree=41934280/67111528
[pid=21814] ppid=21812 vsize=300372 CPUtime=0.87 cores=0,2,4,6
/proc/21814/stat : 21814 (python) S 21812 21814 21157 0 -1 4202496 10982 931 1 0 74 13 0 0 19 0 3 0 670632849 307580928 7415 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/21814/statm: 75093 7415 1880 364 0 11008 0
[pid=21820] ppid=21814 vsize=290116 CPUtime=0.4 cores=0,2,4,6
/proc/21820/stat : 21820 (python) S 21814 21814 21157 0 -1 4202560 1092 0 0 0 21 19 0 0 19 0 2 0 670632957 297078784 5942 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/21820/statm: 72529 5942 410 364 0 8447 0
[pid=21822] ppid=21820 vsize=1018076 CPUtime=99.43 cores=0,2,4,6
/proc/21822/stat : 21822 (gnovelty+2) T 21820 21822 21822 0 -1 4202496 530 0 0 0 9943 0 0 0 25 0 1 0 670632957 1042509824 267 33554432000 134512640 135663704 4290551664 18446744073709551615 134525688 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/21822/statm: 254519 267 113 282 0 254235 0
[pid=21820/tid=21830] ppid=21814 vsize=290116 CPUtime=0 cores=0,2,4,6
/proc/21820/task/21830/stat : 21830 (python) S 21814 21814 21157 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 670642901 297078784 5942 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 514 18446604448689566656 0 0 -1 6 0 0 0
[pid=21831] ppid=21814 vsize=290128 CPUtime=0.23 cores=0,2,4,6

################
# More data... #
################

Current children cumulated CPU time (s) 5000.11
Current children cumulated vsize (KiB) 11813728

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+4978.3 s]
/proc/loadavg: 2.25 2.15 2.05 3/626 22172
/proc/meminfo: memFree=27180352/32950928 swapFree=41934280/67111528
[pid=21814] ppid=21812 vsize=423644 CPUtime=1.49 cores=0,2,4,6
/proc/21814/stat : 21814 (python) S 21812 21814 21157 0 -1 4202496 23562 931 1 0 132 17 0 0 18 0 15 0 670632849 433811456 7531 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/21814/statm: 105911 7531 1904 364 0 41790 0
[pid=21820] ppid=21814 vsize=290116 CPUtime=0.4 cores=0,2,4,6
/proc/21820/stat : 21820 (python) S 21814 21814 21157 0 -1 4202560 1092 0 0 0 21 19 0 0 19 0 2 0 670632957 297078784 5942 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/21820/statm: 72529 5942 410 364 0 8447 0
[pid=21822] ppid=21820 vsize=1018076 CPUtime=99.43 cores=0,2,4,6
/proc/21822/stat : 21822 (gnovelty+2) T 21820 21822 21822 0 -1 4202496 530 0 0 0 9943 0 0 0 25 0 1 0 670632957 1042509824 267 33554432000 134512640 135663704 4290551664 18446744073709551615 134525688 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/21822/statm: 254519 267 113 282 0 254235 0
[pid=21820/tid=21830] ppid=21814 vsize=290116 CPUtime=0 cores=0,2,4,6
/proc/21820/task/21830/stat : 21830 (python) S 21814 21814 21157 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 670642901 297078784 5942 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 514 18446604448689566656 0 0 -1 6 0 0 0
[pid=21831] ppid=21814 vsize=290128 CPUtime=0.4 cores=0,2,4,6
/proc/21831/stat : 21831 (python) S 21814 21814 21157 0 -1 4202560 1085 0 0 0 21 19 0 0 19 0 2 0 670642904 297091072 5932 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/21831/statm: 72532 5932 400 364 0 8447 0
[pid=21833] ppid=21831 vsize=1018076 CPUtime=100.28 cores=0,2,4,6
/proc/21833/stat : 21833 (gnovelty+2) T 21831 21833 21833 0 -1 4202496 531 0 0 0 10028 0 0 0 25 0 1 0 670642904 1042509824 268 33554432000 134512640 135663704 4286986000 18446744073709551615 134524888 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/21833/statm: 254519 268 113 282 0 254235 0
[pid=21831/tid=21850] ppid=21814 vsize=290128 CPUtime=0 cores=0,2,4,6
/proc/21831/task/21850/stat : 21850 (python) S 21814 21814 21157 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 670652944 297091072 5932 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 514 18446604453169769152 0 0 -1 4 0 0 0
[pid=21851] ppid=21814 vsize=300384 CPUtime=0.85 cores=0,2,4,6
/proc/21851/stat : 21851 (python) S 21814 21814 21157 0 -1 4202560 1134 0 0 0 45 40 0 0 18 0 2 0 670652948 307593216 5936 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/21851/statm: 75096 5936 400 364 0 11008 0
[pid=21853] ppid=21851 vsize=416920 CPUtime=199.81 cores=0,2,4,6
/proc/21853/stat : 21853 (adaptg2wsat2009) T 21851 21853 21853 0 -1 4202496 672 0 0 0 19981 0 0 0 25 0 1 0 670652948 426926080 405 33554432000 134512640 135107411 4289665936 18446744073709551615 134531779 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/21853/statm: 104230 405 59 146 0 103106 0
[pid=21851/tid=21860] ppid=21814 vsize=300384 CPUtime=0 cores=0,2,4,6
/proc/21851/task/21860/stat : 21860 (python) S 21814 21814 21157 0 -1 4202560 50 0 0 0 0 0 0 0 18 0 2 0 670662988 307593216 5936 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 514 18446604441936149504 0 0 -1 6 0 0 0
[pid=21861] ppid=21814 vsize=310640 CPUtime=0.43 cores=0,2,4,6
/proc/21861/stat : 21861 (python) S 21814 21814 21157 0 -1 4202560 1104 0 0 0 25 18 0 0 19 0 2 0 670662991 318095360 5940 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/21861/statm: 77660 5940 400 364 0 13569 0
[pid=21863] ppid=21861 vsize=416920 CPUtime=99.44 cores=0,2,4,6
/proc/21863/stat : 21863 (adaptg2wsat2009) T 21861 21863 21863 0 -1 4202496 672 0 0 0 9944 0 0 0 25 0 1 0 670662991 426926080 405 33554432000 134512640 135107411 4291787632 18446744073709551615 134531935 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/21863/statm: 104230 405 59 146 0 103106 0
[pid=21861/tid=21874] ppid=21814 vsize=310640 CPUtime=0 cores=0,2,4,6
/proc/21861/task/21874/stat : 21874 (python) S 21814 21814 21157 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 670672936 318095360 5940 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 514 18446604441607411648 0 0 -1 6 0 0 0
[pid=21875] ppid=21814 vsize=320896 CPUtime=0.42 cores=0,2,4,6
/proc/21875/stat : 21875 (python) S 21814 21814 21157 0 -1 4202560 1123 0 0 0 22 20 0 0 19 0 2 0 670672940 328597504 5944 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/21875/statm: 80224 5944 400 364 0 16130 0
[pid=21877] ppid=21875 vsize=416920 CPUtime=100.39 cores=0,2,4,6
/proc/21877/stat : 21877 (adaptg2wsat2009) T 21875 21877 21877 0 -1 4202496 677 0 0 0 10039 0 0 0 25 0 1 0 670672940 426926080 407 33554432000 134512640 135107411 4289898272 18446744073709551615 134532128 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/21877/statm: 104230 407 59 146 0 103106 0
[pid=21875/tid=21881] ppid=21814 vsize=320896 CPUtime=0 cores=0,2,4,6
/proc/21875/task/21881/stat : 21881 (python) S 21814 21814 21157 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 670682981 328597504 5944 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 514 18446604441607409856 0 0 -1 0 0 0 0
[pid=21882] ppid=21814 vsize=331152 CPUtime=0.42 cores=0,2,4,6
/proc/21882/stat : 21882 (python) S 21814 21814 21157 0 -1 4202560 1137 0 0 0 20 22 0 0 19 0 2 0 670682985 339099648 5948 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/21882/statm: 82788 5948 400 364 0 18691 0
[pid=21884] ppid=21882 vsize=1018076 CPUtime=97.46 cores=0,2,4,6
/proc/21884/stat : 21884 (gnovelty+2) T 21882 21884 21884 0 -1 4202496 535 0 0 0 9746 0 0 0 25 0 1 0 670682985 1042509824 267 33554432000 134512640 135663704 4292431408 18446744073709551615 134525255 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/21884/statm: 254519 267 113 282 0 254235 0
[pid=21882/tid=21891] ppid=21814 vsize=331152 CPUtime=0 cores=0,2,4,6
/proc/21882/task/21891/stat : 21891 (python) S 21814 21814 21157 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 670692731 339099648 5948 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 514 18446604441422357504 0 0 -1 6 0 0 0
[pid=21892] ppid=21814 vsize=341408 CPUtime=0.43 cores=0,2,4,6
/proc/21892/stat : 21892 (python) S 21814 21814 21157 0 -1 4202560 1134 0 0 0 25 18 0 0 19 0 2 0 670692735 349601792 5952 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/21892/statm: 85352 5952 400 364 0 21252 0
[pid=21894] ppid=21892 vsize=247072 CPUtime=99.43 cores=0,2,4,6
/proc/21894/stat : 21894 (FH) T 21892 21894 21894 0 -1 4202496 631 0 0 0 9943 0 0 0 25 0 1 0 670692735 253001728 365 33554432000 134512640 134936988 4292058144 18446744073709551615 134552382 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/21894/statm: 61768 365 41 104 0 61272 0
[pid=21892/tid=21902] ppid=21814 vsize=341408 CPUtime=0 cores=0,2,4,6
/proc/21892/task/21902/stat : 21902 (python) S 21814 21814 21157 0 -1 4202560 51 0 0 0 0 0 0 0 19 0 2 0 670702679 349601792 5952 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 514 18446604447148155200 0 0 -1 6 0 0 0
[pid=21903] ppid=21814 vsize=351664 CPUtime=10.09 cores=0,2,4,6
/proc/21903/stat : 21903 (python) S 21814 21814 21157 0 -1 4202560 1223 0 0 0 541 468 0 0 16 0 2 0 670702683 360103936 5963 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/21903/statm: 87916 5963 407 364 0 23813 0
[pid=21905] ppid=21903 vsize=14444 CPUtime=2234.57 cores=0,2,4,6
/proc/21905/stat : 21905 (march_hi) T 21903 21905 21905 0 -1 4202496 3555 0 0 0 223454 3 0 0 25 0 1 0 670702683 14790656 2517 33554432000 4194304 4955518 140734032956080 18446744073709551615 4297477 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/21905/statm: 3611 2517 106 186 0 3424 0
[pid=21903/tid=21924] ppid=21814 vsize=351664 CPUtime=0 cores=0,2,4,6
/proc/21903/task/21924/stat : 21924 (python) S 21814 21814 21157 0 -1 4202560 51 0 0 0 0 0 0 0 15 0 2 0 670741864 360103936 5963 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 514 18446604449587555200 0 0 -1 0 0 0 0
[pid=21944] ppid=21814 vsize=361920 CPUtime=0.43 cores=0,2,4,6
/proc/21944/stat : 21944 (python) S 21814 21814 21157 0 -1 4202560 1142 0 0 0 25 18 0 0 19 0 2 0 670780431 370606080 5960 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/21944/statm: 90480 5960 400 364 0 26374 0
[pid=21946] ppid=21944 vsize=416920 CPUtime=100.43 cores=0,2,4,6
/proc/21946/stat : 21946 (adaptg2wsat2009) T 21944 21946 21946 0 -1 4202496 682 0 0 0 10043 0 0 0 25 0 1 0 670780431 426926080 407 33554432000 134512640 135107411 4293587216 18446744073709551615 134531239 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/21946/statm: 104230 407 59 146 0 103106 0
[pid=21944/tid=21949] ppid=21814 vsize=361920 CPUtime=0 cores=0,2,4,6
/proc/21944/task/21949/stat : 21949 (python) S 21814 21814 21157 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 670790474 370606080 5960 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 514 18446604450147083520 0 0 -1 4 0 0 0
[pid=21950] ppid=21814 vsize=372176 CPUtime=3.57 cores=0,2,4,6
/proc/21950/stat : 21950 (python) S 21814 21814 21157 0 -1 4202560 1184 0 0 0 180 177 0 0 15 0 2 0 670790477 381108224 5964 33554432000 4194304 5683644 140736734403616 18446744073709551615 251814260294 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/21950/statm: 93044 5964 400 364 0 28935 0
[pid=21952] ppid=21950 vsize=6252 CPUtime=767.82 cores=0,2,4,6
/proc/21952/stat : 21952 (march_hi) R 21950 21952 21952 0 -1 4202496 2154 0 0 0 76781 1 0 0 25 0 1 0 670790478 6402048 1117 33554432000 4194304 4955518 140733678603232 18446744073709551615 4224471 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/21952/statm: 1563 1117 106 186 0 1376 0
[pid=21950/tid=21964] ppid=21814 vsize=372176 CPUtime=0 cores=0,2,4,6
/proc/21950/task/21964/stat : 21964 (python) S 21814 21814 21157 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 670829415 381108224 5964 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 514 18446604447148156992 0 0 -1 0 0 0 0
[pid=21973] ppid=21814 vsize=382432 CPUtime=3.15 cores=0,2,4,6
/proc/21973/stat : 21973 (python) S 21814 21814 21157 0 -1 4202560 1181 0 0 0 172 143 0 0 15 0 2 0 670858877 391610368 5977 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/21973/statm: 95608 5977 409 364 0 31496 0
[pid=21975] ppid=21973 vsize=5100 CPUtime=681.55 cores=0,2,4,6
/proc/21975/stat : 21975 (march_hi) T 21973 21975 21975 0 -1 4202496 2078 0 0 0 68155 0 0 0 25 0 1 0 670858878 5222400 1044 33554432000 4194304 4955518 140734668916592 18446744073709551615 4297477 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/21975/statm: 1275 1044 106 186 0 1088 0
[pid=21973/tid=21986] ppid=21814 vsize=382432 CPUtime=0 cores=0,2,4,6
/proc/21973/task/21986/stat : 21986 (python) S 21814 21814 21157 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 670897904 391610368 5977 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 514 18446604453169766464 0 0 -1 6 0 0 0
[pid=22035] ppid=21814 vsize=392688 CPUtime=0.42 cores=0,2,4,6
/proc/22035/stat : 22035 (python) S 21814 21814 21157 0 -1 4202560 1137 0 0 0 22 20 0 0 19 0 2 0 670927040 402112512 5972 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/22035/statm: 98172 5972 400 364 0 34057 0
[pid=22037] ppid=22035 vsize=247064 CPUtime=99.41 cores=0,2,4,6
/proc/22037/stat : 22037 (FH) T 22035 22037 22037 0 -1 4202496 636 0 0 0 9941 0 0 0 25 0 1 0 670927041 252993536 365 33554432000 134512640 134936988 4293047312 18446744073709551615 134553554 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/22037/statm: 61766 365 41 104 0 61270 0
[pid=22035/tid=22083] ppid=21814 vsize=392688 CPUtime=0 cores=0,2,4,6
/proc/22035/task/22083/stat : 22083 (python) S 21814 21814 21157 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 670936984 402112512 5972 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 514 18446604446229066880 0 0 -1 4 0 0 0
[pid=22087] ppid=21814 vsize=402944 CPUtime=0.42 cores=0,2,4,6
/proc/22087/stat : 22087 (python) S 21814 21814 21157 0 -1 4202560 1146 0 0 0 23 19 0 0 19 0 2 0 670946934 412614656 5976 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/22087/statm: 100736 5976 400 364 0 36618 0
[pid=22089] ppid=22087 vsize=1018076 CPUtime=98.45 cores=0,2,4,6
/proc/22089/stat : 22089 (gnovelty+2) T 22087 22089 22089 0 -1 4202496 534 0 0 0 9845 0 0 0 25 0 1 0 670946934 1042509824 266 33554432000 134512640 135663704 4290247728 18446744073709551615 134524921 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/22089/statm: 254519 266 113 282 0 254235 0
[pid=22087/tid=22093] ppid=21814 vsize=402944 CPUtime=0 cores=0,2,4,6
/proc/22087/task/22093/stat : 22093 (python) S 21814 21814 21157 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 670956779 412614656 5976 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 514 18446604451967739072 0 0 -1 6 0 0 0
[pid=22094] ppid=21814 vsize=413200 CPUtime=0.85 cores=0,2,4,6
/proc/22094/stat : 22094 (python) S 21814 21814 21157 0 -1 4202560 1180 0 0 0 47 38 0 0 19 0 2 0 670956781 423116800 5980 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826066891 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/22094/statm: 103300 5980 400 364 0 39179 0
[pid=22096] ppid=22094 vsize=268420 CPUtime=197.87 cores=0,2,4,6
/proc/22096/stat : 22096 (hybridGM3) T 22094 22096 22096 0 -1 4202496 4901 0 0 0 19787 0 0 0 25 0 1 0 670956782 274862080 4628 33554432000 134512640 135204385 4293217600 18446744073709551615 134536993 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/22096/statm: 67105 4628 64 169 0 66935 0
[pid=22094/tid=22106] ppid=21814 vsize=413200 CPUtime=0 cores=0,2,4,6
/proc/22094/task/22106/stat : 22106 (python) S 21814 21814 21157 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 670976569 423116800 5980 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 514 18446604451686303936 0 0 -1 6 0 0 0
[pid=21814/tid=21821] ppid=21812 vsize=423644 CPUtime=0 cores=0,2,4,6
/proc/21814/task/21821/stat : 21821 (python) S 21812 21814 21157 0 -1 4202560 32 931 0 0 0 0 0 0 20 0 15 0 670632957 433811456 7531 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 2 18446604452299776960 0 0 -1 2 0 0 0
[pid=21814/tid=21832] ppid=21812 vsize=423644 CPUtime=0 cores=0,2,4,6
/proc/21814/task/21832/stat : 21832 (python) S 21812 21814 21157 0 -1 4202560 26 931 0 0 0 0 0 0 20 0 15 0 670642904 433811456 7531 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 2 18446604452299776960 0 0 -1 4 0 0 0
[pid=21814/tid=21852] ppid=21812 vsize=423644 CPUtime=0 cores=0,2,4,6
/proc/21814/task/21852/stat : 21852 (python) S 21812 21814 21157 0 -1 4202560 31 931 0 0 0 0 0 0 18 0 15 0 670652948 433811456 7531 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 2 18446604452299776960 0 0 -1 2 0 0 0
[pid=21814/tid=21862] ppid=21812 vsize=423644 CPUtime=0 cores=0,2,4,6
/proc/21814/task/21862/stat : 21862 (python) S 21812 21814 21157 0 -1 4202560 24 931 0 0 0 0 0 0 20 0 15 0 670662991 433811456 7531 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 2 18446604452299776960 0 0 -1 0 0 0 0
[pid=21814/tid=21876] ppid=21812 vsize=423644 CPUtime=0 cores=0,2,4,6
/proc/21814/task/21876/stat : 21876 (python) S 21812 21814 21157 0 -1 4202560 26 931 0 0 0 0 0 0 20 0 15 0 670672940 433811456 7531 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 2 18446604452299776960 0 0 -1 6 0 0 0
[pid=21814/tid=21883] ppid=21812 vsize=423644 CPUtime=0 cores=0,2,4,6
/proc/21814/task/21883/stat : 21883 (python) S 21812 21814 21157 0 -1 4202560 29 931 0 0 0 0 0 0 20 0 15 0 670682985 433811456 7531 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 2 18446604452299776960 0 0 -1 0 0 0 0
[pid=21814/tid=21893] ppid=21812 vsize=423644 CPUtime=0 cores=0,2,4,6
/proc/21814/task/21893/stat : 21893 (python) S 21812 21814 21157 0 -1 4202560 28 931 0 0 0 0 0 0 20 0 15 0 670692735 433811456 7531 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 2 18446604452299776960 0 0 -1 0 0 0 0
[pid=21814/tid=21904] ppid=21812 vsize=423644 CPUtime=0 cores=0,2,4,6
/proc/21814/task/21904/stat : 21904 (python) S 21812 21814 21157 0 -1 4202560 30 931 0 0 0 0 0 0 15 0 15 0 670702683 433811456 7531 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 2 18446604452299776960 0 0 -1 0 0 0 0
[pid=21814/tid=21945] ppid=21812 vsize=423644 CPUtime=0 cores=0,2,4,6
/proc/21814/task/21945/stat : 21945 (python) S 21812 21814 21157 0 -1 4202560 31 931 0 0 0 0 0 0 20 0 15 0 670780431 433811456 7531 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 2 18446604452299776960 0 0 -1 6 0 0 0
[pid=21814/tid=21951] ppid=21812 vsize=423644 CPUtime=0 cores=0,2,4,6
/proc/21814/task/21951/stat : 21951 (python) S 21812 21814 21157 0 -1 4202560 32 931 0 0 0 0 0 0 18 0 15 0 670790478 433811456 7531 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 2 18446604452299776960 0 0 -1 4 0 0 0
[pid=21814/tid=21974] ppid=21812 vsize=423644 CPUtime=0 cores=0,2,4,6
/proc/21814/task/21974/stat : 21974 (python) S 21812 21814 21157 0 -1 4202560 26 931 0 0 0 0 0 0 18 0 15 0 670858877 433811456 7531 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 2 18446604452299776960 0 0 -1 4 0 0 0
[pid=21814/tid=22036] ppid=21812 vsize=423644 CPUtime=0 cores=0,2,4,6
/proc/21814/task/22036/stat : 22036 (python) S 21812 21814 21157 0 -1 4202560 28 931 0 0 0 0 0 0 20 0 15 0 670927041 433811456 7531 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 2 18446604452299776960 0 0 -1 0 0 0 0
[pid=21814/tid=22088] ppid=21812 vsize=423644 CPUtime=0 cores=0,2,4,6
/proc/21814/task/22088/stat : 22088 (python) S 21812 21814 21157 0 -1 4202560 31 931 0 0 0 0 0 0 20 0 15 0 670946934 433811456 7531 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 2 18446604452299776960 0 0 -1 0 0 0 0
[pid=21814/tid=22095] ppid=21812 vsize=423644 CPUtime=0 cores=0,2,4,6
/proc/21814/task/22095/stat : 22095 (python) S 21812 21814 21157 0 -1 4202560 29 931 0 0 0 0 0 0 20 0 15 0 670956781 433811456 7531 33554432000 4194304 5683644 140736734403616 18446744073709551615 251826063793 0 0 16781312 2 18446604452299776960 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 5000.11
Current children cumulated vsize (KiB) 11813728

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 21814 and gives
#  childrusage.ru_utime.tv_sec=1176
#  childrusage.ru_utime.tv_usec=909082
#  childrusage.ru_stime.tv_sec=10
#  childrusage.ru_stime.tv_usec=689374
# CPU time returned by wait4() is 1187.6
# while last known CPU time is 5000.11
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 4978.4
CPU time (s): 5000.11
CPU user time (s): 4989.51
CPU system time (s): 10.6
CPU usage (%): 100.436
Max. virtual memory (cumulated for all children) (KiB): 11813728

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1176.91
system time used= 10.6894
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 52346
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 9549
involuntary context switches= 2450

runsolver used 43.1684 second user time and 87.0168 second system time

The end

Launcher Data 

Begin job on node126 at 2011-05-17 01:59:01
IDJOB=3386675
IDBENCH=82740
IDSOLVER=1872
FILE ID=node126/3386675-1305590341
RUNJOBID= node126-1305586146-21175
PBS_JOBID= 13350352
Free space on /tmp= 73096 MiB

SOLVER NAME= borg-sat 11.03.02-seq-random (fixed)
BENCH NAME= SAT11/random/medium/unif-k7-r89-v70-c6230-S1254276596-021.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3386675-1305590341/watcher-3386675-1305590341 -o /tmp/evaluation-result-3386675-1305590341/solver-3386675-1305590341 -C 5000 -W 5100 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3386675-1305590341.cnf -seed 1072918822 -budget 5000

TIME LIMIT= 5000 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 38eb341db913fde0f11427c4d808f2c9
RANDOM SEED=1072918822

node126.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      27717968 kB
Buffers:        234872 kB
Cached:        2592400 kB
SwapCached:      49648 kB
Active:        2328644 kB
Inactive:      2605752 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      27717968 kB
SwapTotal:    67111528 kB
SwapFree:     41934280 kB
Dirty:           31868 kB
Writeback:           0 kB
AnonPages:     2070392 kB
Mapped:          28768 kB
Slab:           126864 kB
PageTables:     101748 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 167878852 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73052 MiB
End job on node126 at 2011-05-17 03:22:02