Trace number 3386625

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-seq-random (fixed)? (MO) 3641.13 3627.84

General information on the benchmark

Namerandom/large/
unif-k5-r20-v1000-c20000-S519298227-080-UNKNOWN.cnf
MD5SUMb0f2a8c1ae428dae6bc946bb38e8786d
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark30.3944
Satisfiable
(Un)Satisfiability was proved
Number of variables1000
Number of clauses20000
Sum of the clauses size100000
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 520000
Number of clauses of size over 50

Solver Data 

0.34/0.70	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.34/0.70	c INFO: solving HOME/instance-3386625-1305589674.cnf
0.34/0.70	c INFO: computing task features
0.51/0.80	c INFO: computed variable-clause graph statistics
0.51/0.80	c INFO: computed clause balance statistics
1.01/1.32	c INFO: computed variable balance statistics
1.31/1.61	c INFO: computed small-clause counts
1.31/1.61	c INFO: computed Horn-clause counts
1.31/1.63	c INFO: computed variable graph statistics
1.31/1.63	c INFO: collected features for HOME/instance-3386625-1305589674.cnf in 0.93 s
1.31/1.66	c INFO: running gnovelty+2@0 for 97 with 4999 remaining (b = 0.39)
102.02/102.08	c INFO: running gnovelty+2@0 for 97 with 4898 remaining (b = 0.14)
200.89/200.53	c INFO: running adaptg2wsat2009++@0 for 97 with 4800 remaining (b = 0.09)
301.63/300.94	c INFO: running adaptg2wsat2009++@0 for 97 with 4699 remaining (b = 0.08)
402.31/401.34	c INFO: running adaptg2wsat2009++@0 for 97 with 4599 remaining (b = 0.06)
503.08/501.74	c INFO: running gnovelty+2@0 for 97 with 4499 remaining (b = 0.05)
600.94/599.21	c INFO: running FH@0 for 97 with 4401 remaining (b = 0.04)
700.68/698.64	c INFO: running adaptg2wsat2009++@0 for 97 with 4302 remaining (b = 0.04)
799.41/797.07	c INFO: running gnovelty+2@100 for 97 with 4203 remaining (b = 0.03)
897.19/894.49	c INFO: running adaptg2wsat2009++@0 for 97 with 4106 remaining (b = 0.03)
997.09/993.92	c INFO: running FH@0 for 97 with 4007 remaining (b = 0.03)
1096.85/1093.38	c INFO: running gnovelty+2@0 for 97 with 3907 remaining (b = 0.02)
1196.74/1192.84	c INFO: running adaptg2wsat2009++@0 for 97 with 3808 remaining (b = 0.02)
1295.44/1291.28	c INFO: running hybridGM3@0 for 194 with 3709 remaining (b = 0.04)
1491.81/1487.12	c INFO: running FH@0 for 97 with 3514 remaining (b = 0.02)
1591.55/1586.58	c INFO: running hybridGM3@0 for 194 with 3414 remaining (b = 0.03)
1787.37/1781.51	c INFO: running hybridGM3@0 for 194 with 3220 remaining (b = 0.03)
1982.92/1976.38	c INFO: running TNM@0 for 1455 with 3025 remaining (b = 0.19)
3444.98/3432.49	c INFO: running TNM@1456 for 97 with 1569 remaining (b = 0.01)
3542.93/3529.93	c INFO: running adaptg2wsat2009++@100 for 97 with 1471 remaining (b = 0.01)
3640.70/3627.36	c INFO: running iPAWS@0 for 97 with 1374 remaining (b = 0.01)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3386625-1305589674/watcher-3386625-1305589674 -o /tmp/evaluation-result-3386625-1305589674/solver-3386625-1305589674 -C 5000 -W 5100 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3386625-1305589674.cnf -seed 448214935 -budget 5000 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 5030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.95 2.03 2.00 3/493 9013
/proc/meminfo: memFree=11248184/32951124 swapFree=65980224/67111528
[pid=9013] ppid=9011 vsize=15752 CPUtime=0 cores=1,3,5,7
/proc/9013/stat : 9013 (python) S 9011 9013 8584 0 -1 4202496 839 0 0 0 0 0 0 0 17 0 1 0 651382873 16130048 723 33554432000 4194304 5683644 140734350226976 18446744073709551615 228095449088 0 2147483391 16781312 2 18446744071699207358 0 0 17 5 0 0 0
/proc/9013/statm: 3938 723 334 364 0 425 0

[startup+0.08704 s]
/proc/loadavg: 1.95 2.03 2.00 3/493 9013
/proc/meminfo: memFree=11248184/32951124 swapFree=65980224/67111528
[pid=9013] ppid=9011 vsize=50336 CPUtime=0.04 cores=1,3,5,7
/proc/9013/stat : 9013 (python) R 9011 9013 8584 0 -1 4202496 2005 0 0 0 3 1 0 0 17 0 1 0 651382873 51544064 1765 33554432000 4194304 5683644 140734350226976 18446744073709551615 5116756 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/9013/statm: 12584 1765 480 364 0 1284 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 50336

[startup+0.101046 s]
/proc/loadavg: 1.95 2.03 2.00 3/493 9013
/proc/meminfo: memFree=11248184/32951124 swapFree=65980224/67111528
[pid=9013] ppid=9011 vsize=85364 CPUtime=0.06 cores=1,3,5,7
/proc/9013/stat : 9013 (python) R 9011 9013 8584 0 -1 4202496 2595 0 0 0 4 2 0 0 17 0 1 0 651382873 87412736 2315 33554432000 4194304 5683644 140734350226976 18446744073709551615 4286064 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/9013/statm: 21341 2316 713 364 0 1558 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 85364

[startup+0.301038 s]
/proc/loadavg: 1.95 2.03 2.00 3/493 9013
/proc/meminfo: memFree=11248184/32951124 swapFree=65980224/67111528
[pid=9013] ppid=9011 vsize=135748 CPUtime=0.18 cores=1,3,5,7
/proc/9013/stat : 9013 (python) R 9011 9013 8584 0 -1 4202496 5969 935 1 0 13 5 0 0 17 0 1 0 651382873 139005952 4149 33554432000 4194304 5683644 140734350226976 18446744073709551615 4620076 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/9013/statm: 33937 4149 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 135748

[startup+0.700927 s]
/proc/loadavg: 1.95 2.03 2.00 3/493 9013
/proc/meminfo: memFree=11248184/32951124 swapFree=65980224/67111528
[pid=9013] ppid=9011 vsize=278596 CPUtime=0.41 cores=1,3,5,7
/proc/9013/stat : 9013 (python) R 9011 9013 8584 0 -1 4202496 9248 935 1 0 30 11 0 0 15 0 1 0 651382873 285282304 6921 33554432000 4194304 5683644 140734350226976 18446744073709551615 228095450880 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/9013/statm: 69649 6939 1758 364 0 5573 0
Current children cumulated CPU time (s) 0.41
Current children cumulated vsize (KiB) 278596

[startup+1.50078 s]
/proc/loadavg: 1.95 2.03 2.00 3/494 9018
/proc/meminfo: memFree=11220240/32951124 swapFree=65980224/67111528
[pid=9013] ppid=9011 vsize=284116 CPUtime=1.21 cores=1,3,5,7
/proc/9013/stat : 9013 (python) R 9011 9013 8584 0 -1 4202496 11463 935 1 0 110 11 0 0 16 0 1 0 651382873 290934784 8415 33554432000 4194304 5683644 140734350226976 18446744073709551615 4619584 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/9013/statm: 71029 8415 1827 364 0 6953 0
Current children cumulated CPU time (s) 1.21
Current children cumulated vsize (KiB) 284116

[startup+3.1005 s]
/proc/loadavg: 1.95 2.03 2.00 3/497 9021
/proc/meminfo: memFree=11218224/32951124 swapFree=65980224/67111528
[pid=9013] ppid=9011 vsize=291468 CPUtime=1.38 cores=1,3,5,7
/proc/9013/stat : 9013 (python) S 9011 9013 8584 0 -1 4202496 11841 935 1 0 126 12 0 0 19 0 2 0 651382873 298463232 7740 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107277771 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/9013/statm: 72867 7740 1877 364 0 8785 0
[pid=9019] ppid=9013 vsize=281224 CPUtime=0 cores=1,3,5,7
/proc/9019/stat : 9019 (python) S 9013 9013 8584 0 -1 4202560 729 0 0 0 0 0 0 0 18 0 1 0 651383040 287973376 6213 33554432000 4194304 5683644 140734350226976 18446744073709551615 228095471119 0 0 16781312 514 4713704483983425797 0 0 17 5 0 0 0
/proc/9019/statm: 70306 6213 354 364 0 6224 0
[pid=9021] ppid=9019 vsize=1018676 CPUtime=1.41 cores=1,3,5,7
/proc/9021/stat : 9021 (gnovelty+2) R 9019 9021 9021 0 -1 4202496 775 0 0 0 141 0 0 0 23 0 1 0 651383040 1043124224 510 33554432000 134512640 135663704 4289541936 18446744073709551615 134545644 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/9021/statm: 254669 510 113 282 0 254385 0
[pid=9013/tid=9020] ppid=9011 vsize=291468 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9020/stat : 9020 (python) S 9011 9013 8584 0 -1 4202560 27 935 0 0 0 0 0 0 19 0 2 0 651383040 298463232 7740 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2.79
Current children cumulated vsize (KiB) 1591368

[startup+6.30089 s]
/proc/loadavg: 1.96 2.03 2.00 3/497 9021
/proc/meminfo: memFree=11218632/32951124 swapFree=65980224/67111528
[pid=9013] ppid=9011 vsize=291468 CPUtime=1.38 cores=1,3,5,7
/proc/9013/stat : 9013 (python) S 9011 9013 8584 0 -1 4202496 11841 935 1 0 126 12 0 0 19 0 2 0 651382873 298463232 7740 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107277771 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/9013/statm: 72867 7740 1877 364 0 8785 0
[pid=9019] ppid=9013 vsize=281224 CPUtime=0.02 cores=1,3,5,7
/proc/9019/stat : 9019 (python) S 9013 9013 8584 0 -1 4202560 876 0 0 0 1 1 0 0 18 0 1 0 651383040 287973376 6236 33554432000 4194304 5683644 140734350226976 18446744073709551615 228095471119 0 0 16781312 514 4713704483983425797 0 0 17 5 0 0 0
/proc/9019/statm: 70306 6236 377 364 0 6224 0
[pid=9021] ppid=9019 vsize=1018676 CPUtime=4.61 cores=1,3,5,7
/proc/9021/stat : 9021 (gnovelty+2) R 9019 9021 9021 0 -1 4202496 775 0 0 0 461 0 0 0 25 0 1 0 651383040 1043124224 510 33554432000 134512640 135663704 4289541936 18446744073709551615 134524548 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/9021/statm: 254669 510 113 282 0 254385 0
[pid=9013/tid=9020] ppid=9011 vsize=291468 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9020/stat : 9020 (python) S 9011 9013 8584 0 -1 4202560 27 935 0 0 0 0 0 0 19 0 2 0 651383040 298463232 7740 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 6.01
Current children cumulated vsize (KiB) 1591368

[startup+12.7007 s]
/proc/loadavg: 2.12 2.06 2.01 3/497 9022
/proc/meminfo: memFree=11218052/32951124 swapFree=65980224/67111528
[pid=9013] ppid=9011 vsize=291468 CPUtime=1.38 cores=1,3,5,7
/proc/9013/stat : 9013 (python) S 9011 9013 8584 0 -1 4202496 11841 935 1 0 126 12 0 0 19 0 2 0 651382873 298463232 7740 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107277771 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/9013/statm: 72867 7740 1877 364 0 8785 0
[pid=9019] ppid=9013 vsize=281224 CPUtime=0.03 cores=1,3,5,7
/proc/9019/stat : 9019 (python) S 9013 9013 8584 0 -1 4202560 882 0 0 0 1 2 0 0 18 0 1 0 651383040 287973376 6236 33554432000 4194304 5683644 140734350226976 18446744073709551615 228095471119 0 0 16781312 514 12035518160426729733 0 0 17 5 0 0 0
/proc/9019/statm: 70306 6236 377 364 0 6224 0
[pid=9021] ppid=9019 vsize=1018676 CPUtime=11.01 cores=1,3,5,7
/proc/9021/stat : 9021 (gnovelty+2) R 9019 9021 9021 0 -1 4202496 775 0 0 0 1101 0 0 0 25 0 1 0 651383040 1043124224 510 33554432000 134512640 135663704 4289541936 18446744073709551615 134524882 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/9021/statm: 254669 510 113 282 0 254385 0
[pid=9013/tid=9020] ppid=9011 vsize=291468 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9020/stat : 9020 (python) S 9011 9013 8584 0 -1 4202560 27 935 0 0 0 0 0 0 19 0 2 0 651383040 298463232 7740 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 12.42
Current children cumulated vsize (KiB) 1591368

[startup+25.5004 s]
/proc/loadavg: 2.09 2.06 2.01 3/501 9026
/proc/meminfo: memFree=11190916/32951124 swapFree=65980224/67111528
[pid=9013] ppid=9011 vsize=291468 CPUtime=1.38 cores=1,3,5,7
/proc/9013/stat : 9013 (python) S 9011 9013 8584 0 -1 4202496 11841 935 1 0 126 12 0 0 19 0 2 0 651382873 298463232 7740 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107277771 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/9013/statm: 72867 7740 1877 364 0 8785 0
[pid=9019] ppid=9013 vsize=281224 CPUtime=0.08 cores=1,3,5,7
/proc/9019/stat : 9019 (python) S 9013 9013 8584 0 -1 4202560 889 0 0 0 4 4 0 0 18 0 1 0 651383040 287973376 6236 33554432000 4194304 5683644 140734350226976 18446744073709551615 228095471119 0 0 16781312 514 12035518160426729733 0 0 17 7 0 0 0
/proc/9019/statm: 70306 6236 377 364 0 6224 0
[pid=9021] ppid=9019 vsize=1018676 CPUtime=23.81 cores=1,3,5,7
/proc/9021/stat : 9021 (gnovelty+2) R 9019 9021 9021 0 -1 4202496 775 0 0 0 2381 0 0 0 25 0 1 0 651383040 1043124224 510 33554432000 134512640 135663704 4289541936 18446744073709551615 134524918 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/9021/statm: 254669 510 113 282 0 254385 0
[pid=9013/tid=9020] ppid=9011 vsize=291468 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9020/stat : 9020 (python) S 9011 9013 8584 0 -1 4202560 27 935 0 0 0 0 0 0 19 0 2 0 651383040 298463232 7740 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 25.27
Current children cumulated vsize (KiB) 1591368

[startup+51.1007 s]
/proc/loadavg: 2.06 2.05 2.00 3/501 9027
/proc/meminfo: memFree=11190828/32951124 swapFree=65980224/67111528
[pid=9013] ppid=9011 vsize=291468 CPUtime=1.38 cores=1,3,5,7
/proc/9013/stat : 9013 (python) S 9011 9013 8584 0 -1 4202496 11841 935 1 0 126 12 0 0 19 0 2 0 651382873 298463232 7740 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107277771 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/9013/statm: 72867 7740 1877 364 0 8785 0
[pid=9019] ppid=9013 vsize=281224 CPUtime=0.16 cores=1,3,5,7
/proc/9019/stat : 9019 (python) S 9013 9013 8584 0 -1 4202560 900 0 0 0 7 9 0 0 18 0 1 0 651383040 287973376 6236 33554432000 4194304 5683644 140734350226976 18446744073709551615 228095471119 0 0 16781312 514 12035518160426729733 0 0 17 7 0 0 0
/proc/9019/statm: 70306 6236 377 364 0 6224 0
[pid=9021] ppid=9019 vsize=1018676 CPUtime=49.41 cores=1,3,5,7
/proc/9021/stat : 9021 (gnovelty+2) R 9019 9021 9021 0 -1 4202496 775 0 0 0 4941 0 0 0 25 0 1 0 651383040 1043124224 510 33554432000 134512640 135663704 4289541936 18446744073709551615 134525544 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/9021/statm: 254669 510 113 282 0 254385 0
[pid=9013/tid=9020] ppid=9011 vsize=291468 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9020/stat : 9020 (python) S 9011 9013 8584 0 -1 4202560 27 935 0 0 0 0 0 0 19 0 2 0 651383040 298463232 7740 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.95
Current children cumulated vsize (KiB) 1591368

[startup+102.319 s]
/proc/loadavg: 2.07 2.06 2.00 3/505 9032
/proc/meminfo: memFree=11184908/32951124 swapFree=65980224/67111528
[pid=9013] ppid=9011 vsize=301724 CPUtime=1.41 cores=1,3,5,7
/proc/9013/stat : 9013 (python) S 9011 9013 8584 0 -1 4202496 12701 935 1 0 129 12 0 0 19 0 3 0 651382873 308965376 7764 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107277771 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/9013/statm: 75431 7764 1880 364 0 11346 0
[pid=9019] ppid=9013 vsize=291468 CPUtime=0.35 cores=1,3,5,7
/proc/9019/stat : 9019 (python) S 9013 9013 8584 0 -1 4202560 1079 0 0 0 17 18 0 0 19 0 2 0 651383040 298463232 6274 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/9019/statm: 72867 6274 410 364 0 8785 0
[pid=9021] ppid=9019 vsize=1018676 CPUtime=100.36 cores=1,3,5,7
/proc/9021/stat : 9021 (gnovelty+2) T 9019 9021 9021 0 -1 4202496 775 0 0 0 10036 0 0 0 25 0 1 0 651383040 1043124224 510 33554432000 134512640 135663704 4289541936 18446744073709551615 134544167 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/9021/statm: 254669 510 113 282 0 254385 0
[pid=9019/tid=9029] ppid=9013 vsize=291468 CPUtime=0 cores=1,3,5,7
/proc/9019/task/9029/stat : 9029 (python) S 9013 9013 8584 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 651393079 298463232 6274 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 514 18446604462600122688 0 0 -1 3 0 0 0
[pid=9030] ppid=9013 vsize=291480 CPUtime=0 cores=1,3,5,7
/proc/9030/stat : 9030 (python) S 9013 9013 8584 0 -1 4202560 734 0 0 0 0 0 0 0 19 0 1 0 651393082 298475520 6228 33554432000 4194304 5683644 140734350226976 18446744073709551615 228095471174 0 0 16781312 514 8549054479096348671 0 0 17 3 0 0 0
/proc/9030/statm: 72870 6228 348 364 0 8785 0
[pid=9032] ppid=9030 vsize=1018676 CPUtime=0.22 cores=1,3,5,7
/proc/9032/stat : 9032 (gnovelty+2) R 9030 9032 9032 0 -1 4202496 772 0 0 0 22 0 0 0 21 0 1 0 651393083 1043124224 508 33554432000 134512640 135663704 4291932144 18446744073709551615 134525309 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/9032/statm: 254669 508 113 282 0 254385 0
[pid=9013/tid=9020] ppid=9011 vsize=301724 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9020/stat : 9020 (python) S 9011 9013 8584 0 -1 4202560 27 935 0 0 0 0 0 0 19 0 3 0 651383040 308965376 7764 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 7 0 0 0
[pid=9013/tid=9031] ppid=9011 vsize=301724 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9031/stat : 9031 (python) S 9011 9013 8584 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 3 0 651393082 308965376 7764 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 102.34
Current children cumulated vsize (KiB) 2922024

[startup+162.301 s]
/proc/loadavg: 2.02 2.04 2.00 3/509 9038
/proc/meminfo: memFree=11155944/32951124 swapFree=65980224/67111528
[pid=9013] ppid=9011 vsize=301724 CPUtime=1.41 cores=1,3,5,7
/proc/9013/stat : 9013 (python) S 9011 9013 8584 0 -1 4202496 12701 935 1 0 129 12 0 0 19 0 3 0 651382873 308965376 7764 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107277771 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/9013/statm: 75431 7764 1880 364 0 11346 0
[pid=9019] ppid=9013 vsize=291468 CPUtime=0.35 cores=1,3,5,7
/proc/9019/stat : 9019 (python) S 9013 9013 8584 0 -1 4202560 1079 0 0 0 17 18 0 0 19 0 2 0 651383040 298463232 6274 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/9019/statm: 72867 6274 410 364 0 8785 0
[pid=9021] ppid=9019 vsize=1018676 CPUtime=100.36 cores=1,3,5,7
/proc/9021/stat : 9021 (gnovelty+2) T 9019 9021 9021 0 -1 4202496 775 0 0 0 10036 0 0 0 25 0 1 0 651383040 1043124224 510 33554432000 134512640 135663704 4289541936 18446744073709551615 134544167 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/9021/statm: 254669 510 113 282 0 254385 0
[pid=9019/tid=9029] ppid=9013 vsize=291468 CPUtime=0 cores=1,3,5,7
/proc/9019/task/9029/stat : 9029 (python) S 9013 9013 8584 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 651393079 298463232 6274 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 514 18446604462600122688 0 0 -1 3 0 0 0
[pid=9030] ppid=9013 vsize=291480 CPUtime=0.21 cores=1,3,5,7

################
# More data... #
################

[pid=9048] ppid=9013 vsize=312600 CPUtime=0.37 cores=1,3,5,7
/proc/9048/stat : 9048 (python) S 9013 9013 8584 0 -1 4202560 1119 0 0 0 19 18 0 0 19 0 2 0 651412968 320102400 6424 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/9048/statm: 78150 6424 400 364 0 14059 0
[pid=9050] ppid=9048 vsize=419584 CPUtime=100.26 cores=1,3,5,7
/proc/9050/stat : 9050 (adaptg2wsat2009) T 9048 9050 9050 0 -1 4202496 1471 0 0 0 10026 0 0 0 25 0 1 0 651412968 429654016 1205 33554432000 134512640 135107411 4288189504 18446744073709551615 134531779 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/9050/statm: 104896 1205 59 146 0 103772 0
[pid=9048/tid=9054] ppid=9013 vsize=312600 CPUtime=0 cores=1,3,5,7
/proc/9048/task/9054/stat : 9054 (python) S 9013 9013 8584 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 651423004 320102400 6424 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 514 18446604465517551680 0 0 -1 7 0 0 0
[pid=9055] ppid=9013 vsize=322856 CPUtime=0.37 cores=1,3,5,7
/proc/9055/stat : 9055 (python) S 9013 9013 8584 0 -1 4202560 1124 0 0 0 19 18 0 0 19 0 2 0 651423008 330604544 6428 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/9055/statm: 80714 6428 400 364 0 16620 0
[pid=9057] ppid=9055 vsize=419584 CPUtime=100.36 cores=1,3,5,7
/proc/9057/stat : 9057 (adaptg2wsat2009) T 9055 9057 9057 0 -1 4202496 1469 0 0 0 10036 0 0 0 25 0 1 0 651423008 429654016 1204 33554432000 134512640 135107411 4293798128 18446744073709551615 134531709 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/9057/statm: 104896 1204 59 146 0 103772 0
[pid=9055/tid=9064] ppid=9013 vsize=322856 CPUtime=0 cores=1,3,5,7
/proc/9055/task/9064/stat : 9064 (python) S 9013 9013 8584 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 651433045 330604544 6428 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 514 18446604471158414400 0 0 -1 7 0 0 0
[pid=9065] ppid=9013 vsize=333112 CPUtime=0.37 cores=1,3,5,7
/proc/9065/stat : 9065 (python) S 9013 9013 8584 0 -1 4202560 1133 0 0 0 20 17 0 0 19 0 2 0 651433048 341106688 6432 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/9065/statm: 83278 6432 400 364 0 19181 0
[pid=9067] ppid=9065 vsize=1018676 CPUtime=97.43 cores=1,3,5,7
/proc/9067/stat : 9067 (gnovelty+2) T 9065 9067 9067 0 -1 4202496 775 0 0 0 9743 0 0 0 25 0 1 0 651433048 1043124224 508 33554432000 134512640 135663704 4293775040 18446744073709551615 134524897 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/9067/statm: 254669 508 113 282 0 254385 0
[pid=9065/tid=9071] ppid=9013 vsize=333112 CPUtime=0 cores=1,3,5,7
/proc/9065/task/9071/stat : 9071 (python) S 9013 9013 8584 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 651442792 341106688 6432 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 514 18446604467261773824 0 0 -1 7 0 0 0
[pid=9072] ppid=9013 vsize=343368 CPUtime=0.39 cores=1,3,5,7
/proc/9072/stat : 9072 (python) S 9013 9013 8584 0 -1 4202560 1179 0 0 0 20 19 0 0 19 0 2 0 651442795 351608832 6442 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/9072/statm: 85842 6442 406 364 0 21742 0
[pid=9074] ppid=9072 vsize=249472 CPUtime=99.35 cores=1,3,5,7
/proc/9074/stat : 9074 (FH) T 9072 9074 9074 0 -1 4202496 1401 0 0 0 9935 0 0 0 25 0 1 0 651442796 255459328 1134 33554432000 134512640 134936988 4293178320 18446744073709551615 134554594 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/9074/statm: 62368 1134 41 104 0 61872 0
[pid=9072/tid=9081] ppid=9013 vsize=343368 CPUtime=0 cores=1,3,5,7
/proc/9072/task/9081/stat : 9081 (python) S 9013 9013 8584 0 -1 4202560 97 0 0 0 0 0 0 0 19 0 2 0 651452734 351608832 6442 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 514 18446604469170770368 0 0 -1 7 0 0 0
[pid=9082] ppid=9013 vsize=353624 CPUtime=0.38 cores=1,3,5,7
/proc/9082/stat : 9082 (python) S 9013 9013 8584 0 -1 4202560 1183 0 0 0 20 18 0 0 19 0 2 0 651452738 362110976 6446 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/9082/statm: 88406 6446 406 364 0 24303 0
[pid=9084] ppid=9082 vsize=419584 CPUtime=98.35 cores=1,3,5,7
/proc/9084/stat : 9084 (adaptg2wsat2009) T 9082 9084 9084 0 -1 4202496 1479 0 0 0 9835 0 0 0 25 0 1 0 651452738 429654016 1205 33554432000 134512640 135107411 4292064416 18446744073709551615 134531709 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/9084/statm: 104896 1205 59 146 0 103772 0
[pid=9082/tid=9097] ppid=9013 vsize=353624 CPUtime=0 cores=1,3,5,7
/proc/9082/task/9097/stat : 9097 (python) S 9013 9013 8584 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 651462578 362110976 6446 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 514 18446604458001719552 0 0 -1 7 0 0 0
[pid=9104] ppid=9013 vsize=363880 CPUtime=0.38 cores=1,3,5,7
/proc/9104/stat : 9104 (python) S 9013 9013 8584 0 -1 4202560 1140 0 0 0 20 18 0 0 19 0 2 0 651472323 372613120 6465 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/9104/statm: 90970 6465 400 364 0 26864 0
[pid=9106] ppid=9104 vsize=419584 CPUtime=99.38 cores=1,3,5,7
/proc/9106/stat : 9106 (adaptg2wsat2009) T 9104 9106 9106 0 -1 4202496 1475 0 0 0 9938 0 0 0 25 0 1 0 651472324 429654016 1205 33554432000 134512640 135107411 4288750880 18446744073709551615 134531604 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/9106/statm: 104896 1205 59 146 0 103772 0
[pid=9104/tid=9110] ppid=9013 vsize=363880 CPUtime=0 cores=1,3,5,7
/proc/9104/task/9110/stat : 9110 (python) S 9013 9013 8584 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 651482262 372613120 6465 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 514 18446604469033570688 0 0 -1 7 0 0 0
[pid=9111] ppid=9013 vsize=374136 CPUtime=0.39 cores=1,3,5,7
/proc/9111/stat : 9111 (python) S 9013 9013 8584 0 -1 4202560 1148 0 0 0 21 18 0 0 19 0 2 0 651482266 383115264 6469 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/9111/statm: 93534 6469 400 364 0 29425 0
[pid=9113] ppid=9111 vsize=249472 CPUtime=99.41 cores=1,3,5,7
/proc/9113/stat : 9113 (FH) T 9111 9113 9113 0 -1 4202496 1403 0 0 0 9941 0 0 0 25 0 1 0 651482266 255459328 1134 33554432000 134512640 134936988 4289401792 18446744073709551615 134554229 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/9113/statm: 62368 1134 41 104 0 61872 0
[pid=9111/tid=9120] ppid=9013 vsize=374136 CPUtime=0 cores=1,3,5,7
/proc/9111/task/9120/stat : 9120 (python) S 9013 9013 8584 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 651492208 383115264 6469 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 514 18446604460560172864 0 0 -1 7 0 0 0
[pid=9121] ppid=9013 vsize=384596 CPUtime=0.38 cores=1,3,5,7
/proc/9121/stat : 9121 (python) S 9013 9013 8584 0 -1 4202560 1145 0 0 0 19 19 0 0 19 0 2 0 651492212 393826304 6503 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/9121/statm: 96149 6503 400 364 0 32037 0
[pid=9123] ppid=9121 vsize=1018676 CPUtime=99.42 cores=1,3,5,7
/proc/9123/stat : 9123 (gnovelty+2) T 9121 9123 9123 0 -1 4202496 778 0 0 0 9942 0 0 0 25 0 1 0 651492212 1043124224 509 33554432000 134512640 135663704 4291609072 18446744073709551615 134524945 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/9123/statm: 254669 509 113 282 0 254385 0
[pid=9121/tid=9127] ppid=9013 vsize=384596 CPUtime=0 cores=1,3,5,7
/proc/9121/task/9127/stat : 9127 (python) S 9013 9013 8584 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 651502155 393826304 6503 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 514 18446604469293683776 0 0 -1 7 0 0 0
[pid=9128] ppid=9013 vsize=394996 CPUtime=0.39 cores=1,3,5,7
/proc/9128/stat : 9128 (python) S 9013 9013 8584 0 -1 4202560 1142 0 0 0 20 19 0 0 19 0 2 0 651502158 404475904 6543 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/9128/statm: 98749 6543 400 364 0 34634 0
[pid=9130] ppid=9128 vsize=419584 CPUtime=98.33 cores=1,3,5,7
/proc/9130/stat : 9130 (adaptg2wsat2009) T 9128 9130 9130 0 -1 4202496 1476 0 0 0 9833 0 0 0 25 0 1 0 651502159 429654016 1204 33554432000 134512640 135107411 4286623072 18446744073709551615 134531613 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/9130/statm: 104896 1204 59 146 0 103772 0
[pid=9128/tid=9135] ppid=9013 vsize=394996 CPUtime=0 cores=1,3,5,7
/proc/9128/task/9135/stat : 9135 (python) S 9013 9013 8584 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 651511998 404475904 6543 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 514 18446604466549188608 0 0 -1 7 0 0 0
[pid=9136] ppid=9013 vsize=405252 CPUtime=0.75 cores=1,3,5,7
/proc/9136/stat : 9136 (python) S 9013 9013 8584 0 -1 4202560 1162 0 0 0 38 37 0 0 19 0 2 0 651512002 414978048 6547 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/9136/statm: 101313 6547 400 364 0 37195 0
[pid=9138] ppid=9136 vsize=268420 CPUtime=195.49 cores=1,3,5,7
/proc/9138/stat : 9138 (hybridGM3) T 9136 9138 9138 0 -1 4202496 5446 0 0 0 19549 0 0 0 25 0 1 0 651512002 274862080 5174 33554432000 134512640 135204385 4294659664 18446744073709551615 134538432 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/9138/statm: 67105 5174 63 169 0 66935 0
[pid=9136/tid=9144] ppid=9013 vsize=405252 CPUtime=0 cores=1,3,5,7
/proc/9136/task/9144/stat : 9144 (python) S 9013 9013 8584 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 651531582 414978048 6547 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 514 18446604467261772928 0 0 -1 7 0 0 0
[pid=9145] ppid=9013 vsize=415508 CPUtime=0.4 cores=1,3,5,7
/proc/9145/stat : 9145 (python) S 9013 9013 8584 0 -1 4202560 1161 0 0 0 18 22 0 0 19 0 2 0 651531586 425480192 6564 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/9145/statm: 103877 6564 400 364 0 39756 0
[pid=9147] ppid=9145 vsize=249480 CPUtime=99.4 cores=1,3,5,7
/proc/9147/stat : 9147 (FH) T 9145 9147 9147 0 -1 4202496 1409 0 0 0 9940 0 0 0 25 0 1 0 651531587 255467520 1135 33554432000 134512640 134936988 4293637824 18446744073709551615 134554623 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/9147/statm: 62370 1135 41 104 0 61874 0
[pid=9145/tid=9154] ppid=9013 vsize=415508 CPUtime=0 cores=1,3,5,7
/proc/9145/task/9154/stat : 9154 (python) S 9013 9013 8584 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 651541528 425480192 6564 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 514 18446604455810792384 0 0 -1 1 0 0 0
[pid=9155] ppid=9013 vsize=425764 CPUtime=0.8 cores=1,3,5,7
/proc/9155/stat : 9155 (python) S 9013 9013 8584 0 -1 4202560 1226 0 0 0 40 40 0 0 19 0 2 0 651541532 435982336 6576 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/9155/statm: 106441 6576 408 364 0 42317 0
[pid=9157] ppid=9155 vsize=268424 CPUtime=194.89 cores=1,3,5,7
/proc/9157/stat : 9157 (hybridGM3) T 9155 9157 9157 0 -1 4202496 5447 0 0 0 19489 0 0 0 25 0 1 0 651541532 274866176 5175 33554432000 134512640 135204385 4294645728 18446744073709551615 134538375 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/9157/statm: 67106 5175 63 169 0 66936 0
[pid=9155/tid=9167] ppid=9013 vsize=425764 CPUtime=0 cores=1,3,5,7
/proc/9155/task/9167/stat : 9167 (python) S 9013 9013 8584 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 651561022 435982336 6576 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 514 18446604461275000896 0 0 -1 7 0 0 0
[pid=9168] ppid=9013 vsize=436020 CPUtime=0.78 cores=1,3,5,7
/proc/9168/stat : 9168 (python) S 9013 9013 8584 0 -1 4202560 1221 0 0 0 38 40 0 0 17 0 2 0 651561025 446484480 6579 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/9168/statm: 109005 6579 407 364 0 44878 0
[pid=9170] ppid=9168 vsize=268428 CPUtime=194.82 cores=1,3,5,7
/proc/9170/stat : 9170 (hybridGM3) T 9168 9170 9170 0 -1 4202496 5451 0 0 0 19482 0 0 0 25 0 1 0 651561026 274870272 5175 33554432000 134512640 135204385 4291332672 18446744073709551615 134538016 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/9170/statm: 67107 5175 63 169 0 66937 0
[pid=9168/tid=9184] ppid=9013 vsize=436020 CPUtime=0 cores=1,3,5,7
/proc/9168/task/9184/stat : 9184 (python) S 9013 9013 8584 0 -1 4202560 48 0 0 0 0 0 0 0 17 0 2 0 651580508 446484480 6579 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 514 18446604461275002688 0 0 -1 7 0 0 0
[pid=9185] ppid=9013 vsize=446276 CPUtime=6.33 cores=1,3,5,7
/proc/9185/stat : 9185 (python) S 9013 9013 8584 0 -1 4202560 1226 0 0 0 325 308 0 0 18 0 2 0 651580512 456986624 6590 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/9185/statm: 111569 6590 401 364 0 47439 0
[pid=9187] ppid=9185 vsize=457948 CPUtime=1553.55 cores=1,3,5,7
/proc/9187/stat : 9187 (TNM) T 9185 9187 9187 0 -1 4202496 1392 0 0 0 155355 0 0 0 25 0 1 0 651580512 468938752 1115 33554432000 134512640 135008748 4287112000 18446744073709551615 134527729 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/9187/statm: 114487 1115 46 122 0 113387 0
[pid=9185/tid=9332] ppid=9013 vsize=446276 CPUtime=0 cores=1,3,5,7
/proc/9185/task/9332/stat : 9332 (python) S 9013 9013 8584 0 -1 4202560 51 0 0 0 0 0 0 0 18 0 2 0 651726121 456986624 6590 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 514 18446604464107996160 0 0 -1 7 0 0 0
[pid=9346] ppid=9013 vsize=456684 CPUtime=0 cores=1,3,5,7
/proc/9346/stat : 9346 (python) S 9013 9013 8584 0 -1 4202560 827 0 0 0 0 0 0 0 19 0 1 0 651745610 467644416 6549 33554432000 4194304 5683644 140734350226976 18446744073709551615 228095471174 0 0 16781312 514 4713708332274155519 0 0 17 5 0 0 0
/proc/9346/statm: 114171 6549 345 364 0 50038 0
[pid=9348] ppid=9346 vsize=335996 CPUtime=0.35 cores=1,3,5,7
/proc/9348/stat : 9348 (iPAWS) R 9346 9348 9348 0 -1 4202496 1604 0 0 0 35 0 0 0 21 0 1 0 651745611 344059904 1330 33554432000 134512640 135114423 4293208096 18446744073709551615 134518132 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/9348/statm: 83999 1330 59 147 0 83851 0
[pid=9013/tid=9020] ppid=9011 vsize=466928 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9020/stat : 9020 (python) S 9011 9013 8584 0 -1 4202560 30 935 0 0 0 0 0 0 18 0 19 0 651383040 478134272 8120 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 7 0 0 0
[pid=9013/tid=9031] ppid=9011 vsize=466928 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9031/stat : 9031 (python) S 9011 9013 8584 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 19 0 651393082 478134272 8120 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 3 0 0 0
[pid=9013/tid=9042] ppid=9011 vsize=466928 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9042/stat : 9042 (python) S 9011 9013 8584 0 -1 4202560 33 935 0 0 0 0 0 0 18 0 19 0 651402927 478134272 8120 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 7 0 0 0
[pid=9013/tid=9049] ppid=9011 vsize=466928 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9049/stat : 9049 (python) S 9011 9013 8584 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 19 0 651412968 478134272 8120 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 5 0 0 0
[pid=9013/tid=9056] ppid=9011 vsize=466928 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9056/stat : 9056 (python) S 9011 9013 8584 0 -1 4202560 30 935 0 0 0 0 0 0 20 0 19 0 651423008 478134272 8120 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 3 0 0 0
[pid=9013/tid=9066] ppid=9011 vsize=466928 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9066/stat : 9066 (python) S 9011 9013 8584 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 19 0 651433048 478134272 8120 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 7 0 0 0
[pid=9013/tid=9073] ppid=9011 vsize=466928 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9073/stat : 9073 (python) S 9011 9013 8584 0 -1 4202560 28 935 0 0 0 0 0 0 20 0 19 0 651442796 478134272 8120 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 3 0 0 0
[pid=9013/tid=9083] ppid=9011 vsize=466928 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9083/stat : 9083 (python) S 9011 9013 8584 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 19 0 651452738 478134272 8120 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 3 0 0 0
[pid=9013/tid=9105] ppid=9011 vsize=466928 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9105/stat : 9105 (python) S 9011 9013 8584 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 19 0 651472323 478134272 8120 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 3 0 0 0
[pid=9013/tid=9112] ppid=9011 vsize=466928 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9112/stat : 9112 (python) S 9011 9013 8584 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 19 0 651482266 478134272 8120 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 3 0 0 0
[pid=9013/tid=9122] ppid=9011 vsize=466928 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9122/stat : 9122 (python) S 9011 9013 8584 0 -1 4202560 28 935 0 0 0 0 0 0 20 0 19 0 651492212 478134272 8120 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 5 0 0 0
[pid=9013/tid=9129] ppid=9011 vsize=466928 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9129/stat : 9129 (python) S 9011 9013 8584 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 19 0 651502159 478134272 8120 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 5 0 0 0
[pid=9013/tid=9137] ppid=9011 vsize=466928 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9137/stat : 9137 (python) S 9011 9013 8584 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 19 0 651512002 478134272 8120 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 5 0 0 0
[pid=9013/tid=9146] ppid=9011 vsize=466928 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9146/stat : 9146 (python) S 9011 9013 8584 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 19 0 651531586 478134272 8120 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 7 0 0 0
[pid=9013/tid=9156] ppid=9011 vsize=466928 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9156/stat : 9156 (python) S 9011 9013 8584 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 19 0 651541532 478134272 8120 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 1 0 0 0
[pid=9013/tid=9169] ppid=9011 vsize=466928 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9169/stat : 9169 (python) S 9011 9013 8584 0 -1 4202560 28 935 0 0 0 0 0 0 20 0 19 0 651561026 478134272 8120 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 3 0 0 0
[pid=9013/tid=9186] ppid=9011 vsize=466928 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9186/stat : 9186 (python) S 9011 9013 8584 0 -1 4202560 27 935 0 0 0 0 0 0 18 0 19 0 651580512 478134272 8120 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 1 0 0 0
[pid=9013/tid=9347] ppid=9011 vsize=466928 CPUtime=0 cores=1,3,5,7
/proc/9013/task/9347/stat : 9347 (python) S 9011 9013 8584 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 19 0 651745610 478134272 8120 33554432000 4194304 5683644 140734350226976 18446744073709551615 228107274673 0 0 16781312 2 18446604467261772032 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3641.13
Current children cumulated vsize (KiB) 16060740

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 9013 and gives
#  childrusage.ru_utime.tv_sec=1
#  childrusage.ru_utime.tv_usec=907709
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=165974
# CPU time returned by wait4() is 2.07368
# while last known CPU time is 3641.13
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 3627.84
CPU time (s): 3641.13
CPU user time (s): 3633.93
CPU system time (s): 7.2
CPU usage (%): 100.366
Max. virtual memory (cumulated for all children) (KiB): 16060740

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1.90771
system time used= 0.165974
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 29831
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2931
involuntary context switches= 45

runsolver used 32.4491 second user time and 72.668 second system time

The end

Launcher Data 

Begin job on node142 at 2011-05-17 01:47:54
IDJOB=3386625
IDBENCH=82690
IDSOLVER=1872
FILE ID=node142/3386625-1305589674
RUNJOBID= node142-1305586143-8603
PBS_JOBID= 13350336
Free space on /tmp= 73512 MiB

SOLVER NAME= borg-sat 11.03.02-seq-random (fixed)
BENCH NAME= SAT11/random/large/unif-k5-r20-v1000-c20000-S519298227-080-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3386625-1305589674/watcher-3386625-1305589674 -o /tmp/evaluation-result-3386625-1305589674/solver-3386625-1305589674 -C 5000 -W 5100 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3386625-1305589674.cnf -seed 448214935 -budget 5000

TIME LIMIT= 5000 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= b0f2a8c1ae428dae6bc946bb38e8786d
RANDOM SEED=448214935

node142.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5131.95
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      11248340 kB
Buffers:        316348 kB
Cached:        4055640 kB
SwapCached:      11512 kB
Active:       17473144 kB
Inactive:      3850844 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      11248340 kB
SwapTotal:    67111528 kB
SwapFree:     65980224 kB
Dirty:           32240 kB
Writeback:           0 kB
AnonPages:    16949488 kB
Mapped:          35592 kB
Slab:           236844 kB
PageTables:      71868 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 132035884 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73480 MiB
End job on node142 at 2011-05-17 02:48:24