Trace number 3386438

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-seq-random (fixed)? (MO) 3649.92 3636.64

General information on the benchmark

Namerandom/large/
unif-k5-r20-v1000-c20000-S107009952-009-UNKNOWN.cnf
MD5SUM5093bfaf2b5c3f3d3bcb50ce84a28c32
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark19.882
Satisfiable
(Un)Satisfiability was proved
Number of variables1000
Number of clauses20000
Sum of the clauses size100000
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 520000
Number of clauses of size over 50

Solver Data 

0.39/0.78	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.39/0.78	c INFO: solving HOME/instance-3386438-1305586142.cnf
0.39/0.78	c INFO: computing task features
0.46/0.88	c INFO: computed variable-clause graph statistics
0.46/0.88	c INFO: computed clause balance statistics
0.95/1.38	c INFO: computed variable balance statistics
1.25/1.64	c INFO: computed small-clause counts
1.25/1.64	c INFO: computed Horn-clause counts
1.25/1.66	c INFO: computed variable graph statistics
1.25/1.66	c INFO: collected features for HOME/instance-3386438-1305586142.cnf in 0.88 s
1.25/1.70	c INFO: running gnovelty+2@0 for 97 with 4999 remaining (b = 0.39)
100.98/101.11	c INFO: running gnovelty+2@0 for 97 with 4899 remaining (b = 0.14)
201.85/201.53	c INFO: running adaptg2wsat2009++@0 for 97 with 4799 remaining (b = 0.10)
301.60/300.93	c INFO: running adaptg2wsat2009++@0 for 97 with 4699 remaining (b = 0.08)
401.35/400.33	c INFO: running adaptg2wsat2009++@0 for 97 with 4600 remaining (b = 0.06)
501.99/500.73	c INFO: running gnovelty+2@0 for 97 with 4500 remaining (b = 0.05)
600.70/599.17	c INFO: running FH@0 for 97 with 4401 remaining (b = 0.04)
702.61/700.67	c INFO: running adaptg2wsat2009++@0 for 97 with 4300 remaining (b = 0.04)
800.47/798.11	c INFO: running gnovelty+2@99 for 97 with 4202 remaining (b = 0.03)
896.25/893.53	c INFO: running adaptg2wsat2009++@0 for 97 with 4107 remaining (b = 0.03)
995.98/992.97	c INFO: running FH@0 for 97 with 4008 remaining (b = 0.03)
1095.87/1092.40	c INFO: running gnovelty+2@0 for 97 with 3908 remaining (b = 0.02)
1195.62/1191.85	c INFO: running adaptg2wsat2009++@0 for 97 with 3809 remaining (b = 0.02)
1296.50/1292.30	c INFO: running hybridGM3@0 for 194 with 3708 remaining (b = 0.04)
1493.18/1488.22	c INFO: running FH@0 for 97 with 3512 remaining (b = 0.02)
1593.92/1588.67	c INFO: running hybridGM3@0 for 194 with 3412 remaining (b = 0.03)
1791.60/1785.59	c INFO: running hybridGM3@0 for 194 with 3215 remaining (b = 0.03)
1991.29/1984.49	c INFO: running TNM@0 for 1455 with 3016 remaining (b = 0.19)
3458.20/3445.63	c INFO: running TNM@1460 for 97 with 1555 remaining (b = 0.01)
3551.99/3539.03	c INFO: running adaptg2wsat2009++@99 for 97 with 1462 remaining (b = 0.01)
3649.77/3636.44	c INFO: running TNM@0 for 1364 with 1364 remaining (b = 0.16)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3386438-1305586142/watcher-3386438-1305586142 -o /tmp/evaluation-result-3386438-1305586142/solver-3386438-1305586142 -C 5000 -W 5100 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3386438-1305586142.cnf -seed 2076694856 -budget 5000 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 5030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.00 0.00 0.72 2/474 11471
/proc/meminfo: memFree=14716644/32950928 swapFree=45358800/67111528
[pid=11471] ppid=11469 vsize=15776 CPUtime=0 cores=0,2,4,6
/proc/11471/stat : 11471 (python) S 11469 11471 11438 0 -1 4202496 895 0 0 0 0 0 0 0 25 0 1 0 670210925 16154624 777 33554432000 4194304 5683644 140734649245600 18446744073709551615 254500689920 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/11471/statm: 3944 777 343 364 0 431 0

[startup+0.0448971 s]
/proc/loadavg: 0.00 0.00 0.72 2/474 11471
/proc/meminfo: memFree=14716644/32950928 swapFree=45358800/67111528
[pid=11471] ppid=11469 vsize=18884 CPUtime=0.02 cores=0,2,4,6
/proc/11471/stat : 11471 (python) R 11469 11471 11438 0 -1 4202496 1284 0 0 0 2 0 0 0 25 0 1 0 670210925 19337216 1137 33554432000 4194304 5683644 140734649245600 18446744073709551615 4620076 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/11471/statm: 4721 1137 421 364 0 690 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 18884

[startup+0.100919 s]
/proc/loadavg: 0.00 0.00 0.72 2/474 11471
/proc/meminfo: memFree=14716644/32950928 swapFree=45358800/67111528
[pid=11471] ppid=11469 vsize=85620 CPUtime=0.06 cores=0,2,4,6
/proc/11471/stat : 11471 (python) S 11469 11471 11438 0 -1 4202496 2629 0 0 0 5 1 0 0 18 0 1 0 670210925 87674880 2348 33554432000 4194304 5683644 140734649245600 18446744073709551615 254500689920 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/11471/statm: 21405 2348 713 364 0 1622 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 85620

[startup+0.300889 s]
/proc/loadavg: 0.00 0.00 0.72 2/474 11471
/proc/meminfo: memFree=14716644/32950928 swapFree=45358800/67111528
[pid=11471] ppid=11469 vsize=135752 CPUtime=0.19 cores=0,2,4,6
/proc/11471/stat : 11471 (python) S 11469 11471 11438 0 -1 4202496 5970 934 1 0 14 5 0 0 17 0 1 0 670210925 139010048 4150 33554432000 4194304 5683644 140734649245600 18446744073709551615 254500687877 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/11471/statm: 33938 4150 1032 364 0 3124 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 135752

[startup+0.702958 s]
/proc/loadavg: 0.00 0.00 0.72 2/474 11471
/proc/meminfo: memFree=14716644/32950928 swapFree=45358800/67111528
[pid=11471] ppid=11469 vsize=246708 CPUtime=0.39 cores=0,2,4,6
/proc/11471/stat : 11471 (python) S 11469 11471 11438 0 -1 4202496 8268 934 1 0 28 11 0 0 16 0 1 0 670210925 252628992 6158 33554432000 4194304 5683644 140734649245600 18446744073709551615 254500689920 0 2147483391 16781312 2 18446744071699945619 0 0 17 6 0 0 0
/proc/11471/statm: 61677 6158 1621 364 0 4679 0
Current children cumulated CPU time (s) 0.39
Current children cumulated vsize (KiB) 246708

[startup+1.50063 s]
/proc/loadavg: 0.00 0.00 0.72 3/479 11485
/proc/meminfo: memFree=14648196/32950928 swapFree=45358800/67111528
[pid=11471] ppid=11469 vsize=284120 CPUtime=1.15 cores=0,2,4,6
/proc/11471/stat : 11471 (python) R 11469 11471 11438 0 -1 4202496 11463 934 1 0 103 12 0 0 16 0 1 0 670210925 290938880 8416 33554432000 4194304 5683644 140734649245600 18446744073709551615 4643462 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/11471/statm: 71030 8416 1827 364 0 6954 0
Current children cumulated CPU time (s) 1.15
Current children cumulated vsize (KiB) 284120

[startup+3.10031 s]
/proc/loadavg: 0.00 0.00 0.72 3/482 11488
/proc/meminfo: memFree=14645356/32950928 swapFree=45358800/67111528
[pid=11471] ppid=11469 vsize=291472 CPUtime=1.35 cores=0,2,4,6
/proc/11471/stat : 11471 (python) S 11469 11471 11438 0 -1 4202496 11837 934 1 0 122 13 0 0 18 0 2 0 670210925 298467328 7741 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512518603 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/11471/statm: 72868 7741 1877 364 0 8786 0
[pid=11486] ppid=11471 vsize=281228 CPUtime=0 cores=0,2,4,6
/proc/11486/stat : 11486 (python) S 11471 11471 11438 0 -1 4202560 728 0 0 0 0 0 0 0 15 0 1 0 670211095 287977472 6214 33554432000 4194304 5683644 140734649245600 18446744073709551615 254500711951 0 0 16781312 514 7479379748607459585 0 0 17 2 0 0 0
/proc/11486/statm: 70307 6214 354 364 0 6225 0
[pid=11488] ppid=11486 vsize=1018676 CPUtime=1.39 cores=0,2,4,6
/proc/11488/stat : 11488 (gnovelty+2) R 11486 11488 11488 0 -1 4202496 774 0 0 0 139 0 0 0 21 0 1 0 670211095 1043124224 509 33554432000 134512640 135663704 4287617152 18446744073709551615 134525286 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/11488/statm: 254669 509 113 282 0 254385 0
[pid=11471/tid=11487] ppid=11469 vsize=291472 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11487/stat : 11487 (python) S 11469 11471 11438 0 -1 4202560 25 934 0 0 0 0 0 0 18 0 2 0 670211095 298467328 7741 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2.74
Current children cumulated vsize (KiB) 1591376

[startup+6.30068 s]
/proc/loadavg: 0.16 0.03 0.73 3/485 11491
/proc/meminfo: memFree=14624716/32950928 swapFree=45358800/67111528
[pid=11471] ppid=11469 vsize=291472 CPUtime=1.35 cores=0,2,4,6
/proc/11471/stat : 11471 (python) S 11469 11471 11438 0 -1 4202496 11837 934 1 0 122 13 0 0 18 0 2 0 670210925 298467328 7741 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512518603 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/11471/statm: 72868 7741 1877 364 0 8786 0
[pid=11486] ppid=11471 vsize=281228 CPUtime=0.01 cores=0,2,4,6
/proc/11486/stat : 11486 (python) S 11471 11471 11438 0 -1 4202560 868 0 0 0 1 0 0 0 15 0 1 0 670211095 287977472 6237 33554432000 4194304 5683644 140734649245600 18446744073709551615 254500711951 0 0 16781312 514 7479379748607459585 0 0 17 2 0 0 0
/proc/11486/statm: 70307 6237 377 364 0 6225 0
[pid=11488] ppid=11486 vsize=1018676 CPUtime=4.59 cores=0,2,4,6
/proc/11488/stat : 11488 (gnovelty+2) R 11486 11488 11488 0 -1 4202496 774 0 0 0 459 0 0 0 25 0 1 0 670211095 1043124224 509 33554432000 134512640 135663704 4287617152 18446744073709551615 134524882 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/11488/statm: 254669 509 113 282 0 254385 0
[pid=11471/tid=11487] ppid=11469 vsize=291472 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11487/stat : 11487 (python) S 11469 11471 11438 0 -1 4202560 25 934 0 0 0 0 0 0 18 0 2 0 670211095 298467328 7741 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 5.95
Current children cumulated vsize (KiB) 1591376

[startup+12.7004 s]
/proc/loadavg: 0.31 0.07 0.73 3/485 11491
/proc/meminfo: memFree=14625228/32950928 swapFree=45358800/67111528
[pid=11471] ppid=11469 vsize=291472 CPUtime=1.35 cores=0,2,4,6
/proc/11471/stat : 11471 (python) S 11469 11471 11438 0 -1 4202496 11837 934 1 0 122 13 0 0 18 0 2 0 670210925 298467328 7741 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512518603 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/11471/statm: 72868 7741 1877 364 0 8786 0
[pid=11486] ppid=11471 vsize=281228 CPUtime=0.03 cores=0,2,4,6
/proc/11486/stat : 11486 (python) S 11471 11471 11438 0 -1 4202560 874 0 0 0 2 1 0 0 15 0 1 0 670211095 287977472 6237 33554432000 4194304 5683644 140734649245600 18446744073709551615 254500711951 0 0 16781312 514 317591734659809537 0 0 17 2 0 0 0
/proc/11486/statm: 70307 6237 377 364 0 6225 0
[pid=11488] ppid=11486 vsize=1018676 CPUtime=10.99 cores=0,2,4,6
/proc/11488/stat : 11488 (gnovelty+2) R 11486 11488 11488 0 -1 4202496 774 0 0 0 1099 0 0 0 25 0 1 0 670211095 1043124224 509 33554432000 134512640 135663704 4287617152 18446744073709551615 134525255 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/11488/statm: 254669 509 113 282 0 254385 0
[pid=11471/tid=11487] ppid=11469 vsize=291472 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11487/stat : 11487 (python) S 11469 11471 11438 0 -1 4202560 25 934 0 0 0 0 0 0 18 0 2 0 670211095 298467328 7741 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 12.37
Current children cumulated vsize (KiB) 1591376

[startup+25.5008 s]
/proc/loadavg: 0.68 0.16 0.75 3/485 11491
/proc/meminfo: memFree=14627780/32950928 swapFree=45358800/67111528
[pid=11471] ppid=11469 vsize=291472 CPUtime=1.35 cores=0,2,4,6
/proc/11471/stat : 11471 (python) S 11469 11471 11438 0 -1 4202496 11837 934 1 0 122 13 0 0 18 0 2 0 670210925 298467328 7741 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512518603 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/11471/statm: 72868 7741 1877 364 0 8786 0
[pid=11486] ppid=11471 vsize=281228 CPUtime=0.07 cores=0,2,4,6
/proc/11486/stat : 11486 (python) S 11471 11471 11438 0 -1 4202560 882 0 0 0 5 2 0 0 15 0 1 0 670211095 287977472 6237 33554432000 4194304 5683644 140734649245600 18446744073709551615 254500711951 0 0 16781312 514 317591734659809537 0 0 17 6 0 0 0
/proc/11486/statm: 70307 6237 377 364 0 6225 0
[pid=11488] ppid=11486 vsize=1018676 CPUtime=23.79 cores=0,2,4,6
/proc/11488/stat : 11488 (gnovelty+2) R 11486 11488 11488 0 -1 4202496 774 0 0 0 2379 0 0 0 25 0 1 0 670211095 1043124224 509 33554432000 134512640 135663704 4287617152 18446744073709551615 134525533 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/11488/statm: 254669 509 113 282 0 254385 0
[pid=11471/tid=11487] ppid=11469 vsize=291472 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11487/stat : 11487 (python) S 11469 11471 11438 0 -1 4202560 25 934 0 0 0 0 0 0 18 0 2 0 670211095 298467328 7741 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 25.21
Current children cumulated vsize (KiB) 1591376

[startup+51.1007 s]
/proc/loadavg: 1.13 0.31 0.78 3/485 11492
/proc/meminfo: memFree=14621760/32950928 swapFree=45358800/67111528
[pid=11471] ppid=11469 vsize=291472 CPUtime=1.35 cores=0,2,4,6
/proc/11471/stat : 11471 (python) S 11469 11471 11438 0 -1 4202496 11837 934 1 0 122 13 0 0 18 0 2 0 670210925 298467328 7741 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512518603 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/11471/statm: 72868 7741 1877 364 0 8786 0
[pid=11486] ppid=11471 vsize=281228 CPUtime=0.16 cores=0,2,4,6
/proc/11486/stat : 11486 (python) S 11471 11471 11438 0 -1 4202560 898 0 0 0 9 7 0 0 15 0 1 0 670211095 287977472 6237 33554432000 4194304 5683644 140734649245600 18446744073709551615 254500711951 0 0 16781312 514 7479379748607492095 0 0 17 6 0 0 0
/proc/11486/statm: 70307 6237 377 364 0 6225 0
[pid=11488] ppid=11486 vsize=1018676 CPUtime=49.39 cores=0,2,4,6
/proc/11488/stat : 11488 (gnovelty+2) R 11486 11488 11488 0 -1 4202496 774 0 0 0 4939 0 0 0 25 0 1 0 670211095 1043124224 509 33554432000 134512640 135663704 4287617152 18446744073709551615 134525286 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/11488/statm: 254669 509 113 282 0 254385 0
[pid=11471/tid=11487] ppid=11469 vsize=291472 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11487/stat : 11487 (python) S 11469 11471 11438 0 -1 4202560 25 934 0 0 0 0 0 0 18 0 2 0 670211095 298467328 7741 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 50.9
Current children cumulated vsize (KiB) 1591376

[startup+102.318 s]
/proc/loadavg: 1.62 0.56 0.85 3/489 11567
/proc/meminfo: memFree=14594364/32950928 swapFree=45358800/67111528
[pid=11471] ppid=11469 vsize=301728 CPUtime=1.38 cores=0,2,4,6
/proc/11471/stat : 11471 (python) S 11469 11471 11438 0 -1 4202496 12700 934 1 0 125 13 0 0 19 0 3 0 670210925 308969472 7764 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512518603 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/11471/statm: 75432 7764 1880 364 0 11347 0
[pid=11486] ppid=11471 vsize=291472 CPUtime=0.35 cores=0,2,4,6
/proc/11486/stat : 11486 (python) S 11471 11471 11438 0 -1 4202560 1081 0 0 0 19 16 0 0 17 0 2 0 670211095 298467328 6284 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/11486/statm: 72868 6284 419 364 0 8786 0
[pid=11488] ppid=11486 vsize=1018676 CPUtime=99.37 cores=0,2,4,6
/proc/11488/stat : 11488 (gnovelty+2) T 11486 11488 11488 0 -1 4202496 774 0 0 0 9937 0 0 0 25 0 1 0 670211095 1043124224 509 33554432000 134512640 135663704 4287617152 18446744073709551615 134525286 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/11488/statm: 254669 509 113 282 0 254385 0
[pid=11486/tid=11564] ppid=11471 vsize=291472 CPUtime=0 cores=0,2,4,6
/proc/11486/task/11564/stat : 11564 (python) S 11471 11471 11438 0 -1 4202560 48 0 0 0 0 0 0 0 17 0 2 0 670221033 298467328 6284 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 514 18446604441769220160 0 0 -1 6 0 0 0
[pid=11565] ppid=11471 vsize=291484 CPUtime=0 cores=0,2,4,6
/proc/11565/stat : 11565 (python) S 11471 11471 11438 0 -1 4202560 742 0 0 0 0 0 0 0 18 0 1 0 670221036 298479616 6228 33554432000 4194304 5683644 140734649245600 18446744073709551615 254500712006 0 0 16781312 514 2144794796609208579 0 0 17 0 0 0 0
/proc/11565/statm: 72871 6228 348 364 0 8786 0
[pid=11567] ppid=11565 vsize=1018676 CPUtime=1.2 cores=0,2,4,6
/proc/11567/stat : 11567 (gnovelty+2) R 11565 11567 11567 0 -1 4202496 771 0 0 0 120 0 0 0 25 0 1 0 670221037 1043124224 509 33554432000 134512640 135663704 4287735488 18446744073709551615 134545704 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/11567/statm: 254669 509 113 282 0 254385 0
[pid=11471/tid=11487] ppid=11469 vsize=301728 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11487/stat : 11487 (python) S 11469 11471 11438 0 -1 4202560 25 934 0 0 0 0 0 0 18 0 3 0 670211095 308969472 7764 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 0 0 0 0
[pid=11471/tid=11566] ppid=11469 vsize=301728 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11566/stat : 11566 (python) S 11469 11471 11438 0 -1 4202560 29 934 0 0 0 0 0 0 20 0 3 0 670221037 308969472 7764 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 102.3
Current children cumulated vsize (KiB) 2922036

[startup+162.301 s]
/proc/loadavg: 1.86 0.82 0.92 3/489 11569
/proc/meminfo: memFree=14591668/32950928 swapFree=45358800/67111528
[pid=11471] ppid=11469 vsize=301728 CPUtime=1.38 cores=0,2,4,6
/proc/11471/stat : 11471 (python) S 11469 11471 11438 0 -1 4202496 12700 934 1 0 125 13 0 0 19 0 3 0 670210925 308969472 7764 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512518603 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/11471/statm: 75432 7764 1880 364 0 11347 0
[pid=11486] ppid=11471 vsize=291472 CPUtime=0.35 cores=0,2,4,6
/proc/11486/stat : 11486 (python) S 11471 11471 11438 0 -1 4202560 1081 0 0 0 19 16 0 0 17 0 2 0 670211095 298467328 6284 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/11486/statm: 72868 6284 419 364 0 8786 0
[pid=11488] ppid=11486 vsize=1018676 CPUtime=99.37 cores=0,2,4,6
/proc/11488/stat : 11488 (gnovelty+2) T 11486 11488 11488 0 -1 4202496 774 0 0 0 9937 0 0 0 25 0 1 0 670211095 1043124224 509 33554432000 134512640 135663704 4287617152 18446744073709551615 134525286 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/11488/statm: 254669 509 113 282 0 254385 0
[pid=11486/tid=11564] ppid=11471 vsize=291472 CPUtime=0 cores=0,2,4,6
/proc/11486/task/11564/stat : 11564 (python) S 11471 11471 11438 0 -1 4202560 48 0 0 0 0 0 0 0 17 0 2 0 670221033 298467328 6284 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 514 18446604441769220160 0 0 -1 6 0 0 0
[pid=11565] ppid=11471 vsize=291484 CPUtime=0.21 cores=0,2,4,6

################
# More data... #
################

[pid=11587] ppid=11471 vsize=312600 CPUtime=0.36 cores=0,2,4,6
/proc/11587/stat : 11587 (python) S 11471 11471 11438 0 -1 4202560 1090 0 0 0 19 17 0 0 19 0 2 0 670241019 320102400 6424 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/11587/statm: 78150 6424 400 364 0 14059 0
[pid=11589] ppid=11587 vsize=419584 CPUtime=99.36 cores=0,2,4,6
/proc/11589/stat : 11589 (adaptg2wsat2009) T 11587 11589 11589 0 -1 4202496 1474 0 0 0 9936 0 0 0 25 0 1 0 670241019 429654016 1206 33554432000 134512640 135107411 4294820432 18446744073709551615 134531779 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/11589/statm: 104896 1206 59 146 0 103772 0
[pid=11587/tid=11593] ppid=11471 vsize=312600 CPUtime=0 cores=0,2,4,6
/proc/11587/task/11593/stat : 11593 (python) S 11471 11471 11438 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 670250955 320102400 6424 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 514 18446604448240147520 0 0 -1 2 0 0 0
[pid=11594] ppid=11471 vsize=322856 CPUtime=0.36 cores=0,2,4,6
/proc/11594/stat : 11594 (python) S 11471 11471 11438 0 -1 4202560 1116 0 0 0 20 16 0 0 19 0 2 0 670250959 330604544 6428 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/11594/statm: 80714 6428 400 364 0 16620 0
[pid=11596] ppid=11594 vsize=419584 CPUtime=100.24 cores=0,2,4,6
/proc/11596/stat : 11596 (adaptg2wsat2009) T 11594 11596 11596 0 -1 4202496 1472 0 0 0 10024 0 0 0 25 0 1 0 670250960 429654016 1204 33554432000 134512640 135107411 4294568544 18446744073709551615 134532349 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/11596/statm: 104896 1204 59 146 0 103772 0
[pid=11594/tid=11600] ppid=11471 vsize=322856 CPUtime=0 cores=0,2,4,6
/proc/11594/task/11600/stat : 11600 (python) S 11471 11471 11438 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 670260995 330604544 6428 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 514 18446604441769221056 0 0 -1 4 0 0 0
[pid=11601] ppid=11471 vsize=333112 CPUtime=0.36 cores=0,2,4,6
/proc/11601/stat : 11601 (python) S 11471 11471 11438 0 -1 4202560 1131 0 0 0 19 17 0 0 19 0 2 0 670260999 341106688 6432 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/11601/statm: 83278 6432 400 364 0 19181 0
[pid=11603] ppid=11601 vsize=1018676 CPUtime=98.37 cores=0,2,4,6
/proc/11603/stat : 11603 (gnovelty+2) T 11601 11603 11603 0 -1 4202496 773 0 0 0 9837 0 0 0 25 0 1 0 670260999 1043124224 508 33554432000 134512640 135663704 4293930928 18446744073709551615 134544196 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/11603/statm: 254669 508 113 282 0 254385 0
[pid=11601/tid=11610] ppid=11471 vsize=333112 CPUtime=0 cores=0,2,4,6
/proc/11601/task/11610/stat : 11610 (python) S 11471 11471 11438 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 670270839 341106688 6432 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 514 18446604448975418496 0 0 -1 6 0 0 0
[pid=11611] ppid=11471 vsize=343368 CPUtime=0.39 cores=0,2,4,6
/proc/11611/stat : 11611 (python) S 11471 11471 11438 0 -1 4202560 1179 0 0 0 20 19 0 0 19 0 2 0 670270843 351608832 6442 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/11611/statm: 85842 6442 406 364 0 21742 0
[pid=11613] ppid=11611 vsize=249480 CPUtime=101.46 cores=0,2,4,6
/proc/11613/stat : 11613 (FH) T 11611 11613 11613 0 -1 4202496 1397 0 0 0 10146 0 0 0 25 0 1 0 670270843 255467520 1134 33554432000 134512640 134936988 4292548816 18446744073709551615 134553236 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/11613/statm: 62370 1134 41 104 0 61874 0
[pid=11611/tid=11617] ppid=11471 vsize=343368 CPUtime=0 cores=0,2,4,6
/proc/11611/task/11617/stat : 11617 (python) S 11471 11471 11438 0 -1 4202560 95 0 0 0 0 0 0 0 19 0 2 0 670280989 351608832 6442 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 514 18446604453477115776 0 0 -1 2 0 0 0
[pid=11618] ppid=11471 vsize=353624 CPUtime=0.37 cores=0,2,4,6
/proc/11618/stat : 11618 (python) S 11471 11471 11438 0 -1 4202560 1171 0 0 0 19 18 0 0 19 0 2 0 670280993 362110976 6446 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/11618/statm: 88406 6446 406 364 0 24303 0
[pid=11620] ppid=11618 vsize=419584 CPUtime=97.4 cores=0,2,4,6
/proc/11620/stat : 11620 (adaptg2wsat2009) T 11618 11620 11620 0 -1 4202496 1478 0 0 0 9740 0 0 0 25 0 1 0 670280993 429654016 1205 33554432000 134512640 135107411 4286571616 18446744073709551615 134531246 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/11620/statm: 104896 1205 59 146 0 103772 0
[pid=11618/tid=11636] ppid=11471 vsize=353624 CPUtime=0 cores=0,2,4,6
/proc/11618/task/11636/stat : 11636 (python) S 11471 11471 11438 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 670290734 362110976 6446 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 514 18446604453853311040 0 0 -1 6 0 0 0
[pid=11640] ppid=11471 vsize=363880 CPUtime=0.37 cores=0,2,4,6
/proc/11640/stat : 11640 (python) S 11471 11471 11438 0 -1 4202560 1141 0 0 0 20 17 0 0 19 0 2 0 670300279 372613120 6449 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/11640/statm: 90970 6449 400 364 0 26864 0
[pid=11642] ppid=11640 vsize=419584 CPUtime=99.4 cores=0,2,4,6
/proc/11642/stat : 11642 (adaptg2wsat2009) T 11640 11642 11642 0 -1 4202496 1478 0 0 0 9940 0 0 0 25 0 1 0 670300279 429654016 1205 33554432000 134512640 135107411 4291071760 18446744073709551615 134539270 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/11642/statm: 104896 1205 59 146 0 103772 0
[pid=11640/tid=11652] ppid=11471 vsize=363880 CPUtime=0 cores=0,2,4,6
/proc/11640/task/11652/stat : 11652 (python) S 11471 11471 11438 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 670310219 372613120 6449 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 514 18446604442587020224 0 0 -1 6 0 0 0
[pid=11653] ppid=11471 vsize=374136 CPUtime=0.36 cores=0,2,4,6
/proc/11653/stat : 11653 (python) S 11471 11471 11438 0 -1 4202560 1149 0 0 0 18 18 0 0 19 0 2 0 670310223 383115264 6453 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/11653/statm: 93534 6453 400 364 0 29425 0
[pid=11655] ppid=11653 vsize=249480 CPUtime=99.37 cores=0,2,4,6
/proc/11655/stat : 11655 (FH) T 11653 11655 11655 0 -1 4202496 1403 0 0 0 9937 0 0 0 25 0 1 0 670310223 255467520 1134 33554432000 134512640 134936988 4286761504 18446744073709551615 134553297 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/11655/statm: 62370 1134 41 104 0 61874 0
[pid=11653/tid=11659] ppid=11471 vsize=374136 CPUtime=0 cores=0,2,4,6
/proc/11653/task/11659/stat : 11659 (python) S 11471 11471 11438 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 670320161 383115264 6453 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 514 18446604441335278272 0 0 -1 6 0 0 0
[pid=11660] ppid=11471 vsize=384392 CPUtime=0.37 cores=0,2,4,6
/proc/11660/stat : 11660 (python) S 11471 11471 11438 0 -1 4202560 1141 0 0 0 21 16 0 0 19 0 2 0 670320166 393617408 6481 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/11660/statm: 96098 6481 400 364 0 31986 0
[pid=11662] ppid=11660 vsize=1018676 CPUtime=99.4 cores=0,2,4,6
/proc/11662/stat : 11662 (gnovelty+2) T 11660 11662 11662 0 -1 4202496 778 0 0 0 9940 0 0 0 25 0 1 0 670320166 1043124224 509 33554432000 134512640 135663704 4287383264 18446744073709551615 134524897 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/11662/statm: 254669 509 113 282 0 254385 0
[pid=11660/tid=11669] ppid=11471 vsize=384392 CPUtime=0 cores=0,2,4,6
/proc/11660/task/11669/stat : 11669 (python) S 11471 11471 11438 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 670330107 393617408 6481 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 514 18446604450079061056 0 0 -1 0 0 0 0
[pid=11670] ppid=11471 vsize=394648 CPUtime=0.39 cores=0,2,4,6
/proc/11670/stat : 11670 (python) S 11471 11471 11438 0 -1 4202560 1138 0 0 0 23 16 0 0 19 0 2 0 670330111 404119552 6485 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/11670/statm: 98662 6485 400 364 0 34547 0
[pid=11672] ppid=11670 vsize=419584 CPUtime=100.4 cores=0,2,4,6
/proc/11672/stat : 11672 (adaptg2wsat2009) T 11670 11672 11672 0 -1 4202496 1480 0 0 0 10040 0 0 0 25 0 1 0 670330111 429654016 1206 33554432000 134512640 135107411 4293435920 18446744073709551615 134531600 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/11672/statm: 104896 1206 59 146 0 103772 0
[pid=11670/tid=11676] ppid=11471 vsize=394648 CPUtime=0 cores=0,2,4,6
/proc/11670/task/11676/stat : 11676 (python) S 11471 11471 11438 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 670340152 404119552 6485 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 514 18446604447137599808 0 0 -1 6 0 0 0
[pid=11677] ppid=11471 vsize=405080 CPUtime=0.77 cores=0,2,4,6
/proc/11677/stat : 11677 (python) S 11471 11471 11438 0 -1 4202560 1167 0 0 0 40 37 0 0 19 0 2 0 670340156 414801920 6504 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/11677/statm: 101270 6504 400 364 0 37152 0
[pid=11679] ppid=11677 vsize=268420 CPUtime=195.88 cores=0,2,4,6
/proc/11679/stat : 11679 (hybridGM3) T 11677 11679 11679 0 -1 4202496 5446 0 0 0 19587 1 0 0 25 0 1 0 670340157 274862080 5174 33554432000 134512640 135204385 4289712688 18446744073709551615 134538314 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/11679/statm: 67105 5174 63 169 0 66935 0
[pid=11677/tid=11689] ppid=11471 vsize=405080 CPUtime=0 cores=0,2,4,6
/proc/11677/task/11689/stat : 11689 (python) S 11471 11471 11438 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 670359744 414801920 6504 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 514 18446604442587018432 0 0 -1 6 0 0 0
[pid=11690] ppid=11471 vsize=415336 CPUtime=0.39 cores=0,2,4,6
/proc/11690/stat : 11690 (python) S 11471 11471 11438 0 -1 4202560 1152 0 0 0 20 19 0 0 19 0 2 0 670359748 425304064 6508 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/11690/statm: 103834 6508 400 364 0 39713 0
[pid=11692] ppid=11690 vsize=249472 CPUtime=100.39 cores=0,2,4,6
/proc/11692/stat : 11692 (FH) T 11690 11692 11692 0 -1 4202496 1407 0 0 0 10039 0 0 0 25 0 1 0 670359749 255459328 1134 33554432000 134512640 134936988 4290135312 18446744073709551615 134552080 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/11692/statm: 62368 1134 41 104 0 61872 0
[pid=11690/tid=11700] ppid=11471 vsize=415336 CPUtime=0 cores=0,2,4,6
/proc/11690/task/11700/stat : 11700 (python) S 11471 11471 11438 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 670369788 425304064 6508 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 514 18446604442960353088 0 0 -1 6 0 0 0
[pid=11701] ppid=11471 vsize=425592 CPUtime=0.79 cores=0,2,4,6
/proc/11701/stat : 11701 (python) S 11471 11471 11438 0 -1 4202560 1223 0 0 0 42 37 0 0 19 0 2 0 670369793 435806208 6539 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/11701/statm: 106398 6539 408 364 0 42274 0
[pid=11703] ppid=11701 vsize=268420 CPUtime=196.87 cores=0,2,4,6
/proc/11703/stat : 11703 (hybridGM3) T 11701 11703 11703 0 -1 4202496 5446 0 0 0 19687 0 0 0 25 0 1 0 670369793 274862080 5174 33554432000 134512640 135204385 4291534272 18446744073709551615 134538524 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/11703/statm: 67105 5174 63 169 0 66935 0
[pid=11701/tid=11713] ppid=11471 vsize=425592 CPUtime=0 cores=0,2,4,6
/proc/11701/task/11713/stat : 11713 (python) S 11471 11471 11438 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 670389481 435806208 6539 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 514 18446604441506261888 0 0 -1 0 0 0 0
[pid=11714] ppid=11471 vsize=435848 CPUtime=0.78 cores=0,2,4,6
/proc/11714/stat : 11714 (python) S 11471 11471 11438 0 -1 4202560 1216 0 0 0 42 36 0 0 19 0 2 0 670389484 446308352 6542 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/11714/statm: 108962 6542 407 364 0 44835 0
[pid=11716] ppid=11714 vsize=268424 CPUtime=198.86 cores=0,2,4,6
/proc/11716/stat : 11716 (hybridGM3) T 11714 11716 11716 0 -1 4202496 5450 0 0 0 19885 1 0 0 25 0 1 0 670389485 274866176 5175 33554432000 134512640 135204385 4288174640 18446744073709551615 134538395 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/11716/statm: 67106 5175 63 169 0 66936 0
[pid=11714/tid=11727] ppid=11471 vsize=435848 CPUtime=0 cores=0,2,4,6
/proc/11714/task/11727/stat : 11727 (python) S 11471 11471 11438 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 670409371 446308352 6542 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 514 18446604441335275584 0 0 -1 4 0 0 0
[pid=11728] ppid=11471 vsize=446104 CPUtime=6.2 cores=0,2,4,6
/proc/11728/stat : 11728 (python) S 11471 11471 11438 0 -1 4202560 1211 0 0 0 343 277 0 0 18 0 2 0 670409375 456810496 6540 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/11728/statm: 111526 6540 401 364 0 47396 0
[pid=11730] ppid=11728 vsize=457944 CPUtime=1554.38 cores=0,2,4,6
/proc/11730/stat : 11730 (TNM) T 11728 11730 11730 0 -1 4202496 1391 0 0 0 155437 1 0 0 25 0 1 0 670409375 468934656 1114 33554432000 134512640 135008748 4292512816 18446744073709551615 134527444 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/11730/statm: 114486 1114 46 122 0 113386 0
[pid=11728/tid=11829] ppid=11471 vsize=446104 CPUtime=0 cores=0,2,4,6
/proc/11728/task/11829/stat : 11829 (python) S 11471 11471 11438 0 -1 4202560 48 0 0 0 0 0 0 0 18 0 2 0 670555487 456810496 6540 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 514 18446604441955336384 0 0 -1 0 0 0 0
[pid=11839] ppid=11471 vsize=456752 CPUtime=0 cores=0,2,4,6
/proc/11839/stat : 11839 (python) S 11471 11471 11438 0 -1 4202560 829 0 0 0 0 0 0 0 19 0 1 0 670574570 467714048 6566 33554432000 4194304 5683644 140734649245600 18446744073709551615 254500712006 0 0 16781312 514 8278750540523896831 0 0 17 2 0 0 0
/proc/11839/statm: 114188 6566 345 364 0 50055 0
[pid=11841] ppid=11839 vsize=452768 CPUtime=0.08 cores=0,2,4,6
/proc/11841/stat : 11841 (TNM) R 11839 11841 11841 0 -1 4202496 921 0 0 0 8 0 0 0 20 0 1 0 670574570 463634432 646 33554432000 134512640 135008748 4288239648 18446744073709551615 134523511 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/11841/statm: 113192 646 45 122 0 112092 0
[pid=11471/tid=11487] ppid=11469 vsize=466996 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11487/stat : 11487 (python) S 11469 11471 11438 0 -1 4202560 28 934 0 0 0 0 0 0 18 0 19 0 670211095 478203904 8137 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 0 0 0 0
[pid=11471/tid=11566] ppid=11469 vsize=466996 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11566/stat : 11566 (python) S 11469 11471 11438 0 -1 4202560 29 934 0 0 0 0 0 0 20 0 19 0 670221037 478203904 8137 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 0 0 0 0
[pid=11471/tid=11577] ppid=11469 vsize=466996 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11577/stat : 11577 (python) S 11469 11471 11438 0 -1 4202560 28 934 0 0 0 0 0 0 18 0 19 0 670231079 478203904 8137 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 0 0 0 0
[pid=11471/tid=11588] ppid=11469 vsize=466996 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11588/stat : 11588 (python) S 11469 11471 11438 0 -1 4202560 24 934 0 0 0 0 0 0 20 0 19 0 670241019 478203904 8137 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 0 0 0 0
[pid=11471/tid=11595] ppid=11469 vsize=466996 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11595/stat : 11595 (python) S 11469 11471 11438 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 19 0 670250959 478203904 8137 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 0 0 0 0
[pid=11471/tid=11602] ppid=11469 vsize=466996 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11602/stat : 11602 (python) S 11469 11471 11438 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 19 0 670260999 478203904 8137 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 0 0 0 0
[pid=11471/tid=11612] ppid=11469 vsize=466996 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11612/stat : 11612 (python) S 11469 11471 11438 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 19 0 670270843 478203904 8137 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 6 0 0 0
[pid=11471/tid=11619] ppid=11469 vsize=466996 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11619/stat : 11619 (python) S 11469 11471 11438 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 19 0 670280993 478203904 8137 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 2 0 0 0
[pid=11471/tid=11641] ppid=11469 vsize=466996 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11641/stat : 11641 (python) S 11469 11471 11438 0 -1 4202560 31 934 0 0 0 0 0 0 20 0 19 0 670300279 478203904 8137 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 2 0 0 0
[pid=11471/tid=11654] ppid=11469 vsize=466996 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11654/stat : 11654 (python) S 11469 11471 11438 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 19 0 670310223 478203904 8137 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 0 0 0 0
[pid=11471/tid=11661] ppid=11469 vsize=466996 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11661/stat : 11661 (python) S 11469 11471 11438 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 19 0 670320166 478203904 8137 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 6 0 0 0
[pid=11471/tid=11671] ppid=11469 vsize=466996 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11671/stat : 11671 (python) S 11469 11471 11438 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 19 0 670330111 478203904 8137 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 0 0 0 0
[pid=11471/tid=11678] ppid=11469 vsize=466996 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11678/stat : 11678 (python) S 11469 11471 11438 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 19 0 670340156 478203904 8137 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 0 0 0 0
[pid=11471/tid=11691] ppid=11469 vsize=466996 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11691/stat : 11691 (python) S 11469 11471 11438 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 19 0 670359748 478203904 8137 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 0 0 0 0
[pid=11471/tid=11702] ppid=11469 vsize=466996 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11702/stat : 11702 (python) S 11469 11471 11438 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 19 0 670369793 478203904 8137 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 6 0 0 0
[pid=11471/tid=11715] ppid=11469 vsize=466996 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11715/stat : 11715 (python) S 11469 11471 11438 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 19 0 670389485 478203904 8137 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 6 0 0 0
[pid=11471/tid=11729] ppid=11469 vsize=466996 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11729/stat : 11729 (python) S 11469 11471 11438 0 -1 4202560 27 934 0 0 0 0 0 0 18 0 19 0 670409375 478203904 8137 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 6 0 0 0
[pid=11471/tid=11840] ppid=11469 vsize=466996 CPUtime=0 cores=0,2,4,6
/proc/11471/task/11840/stat : 11840 (python) S 11469 11471 11438 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 19 0 670574570 478203904 8137 33554432000 4194304 5683644 140734649245600 18446744073709551615 254512515505 0 0 16781312 2 18446604453853311936 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3649.92
Current children cumulated vsize (KiB) 16176240

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 11471 and gives
#  childrusage.ru_utime.tv_sec=1
#  childrusage.ru_utime.tv_usec=911709
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=182972
# CPU time returned by wait4() is 2.09468
# while last known CPU time is 3649.92
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 3636.64
CPU time (s): 3649.92
CPU user time (s): 3643.27
CPU system time (s): 6.65
CPU usage (%): 100.365
Max. virtual memory (cumulated for all children) (KiB): 16176240

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1.91171
system time used= 0.182972
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 29424
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 8019
involuntary context switches= 60

runsolver used 35.6056 second user time and 68.9785 second system time

The end

Launcher Data 

Begin job on node149 at 2011-05-17 00:49:02
IDJOB=3386438
IDBENCH=82503
IDSOLVER=1872
FILE ID=node149/3386438-1305586142
RUNJOBID= node149-1305586142-11456
PBS_JOBID= 13350329
Free space on /tmp= 73532 MiB

SOLVER NAME= borg-sat 11.03.02-seq-random (fixed)
BENCH NAME= SAT11/random/large/unif-k5-r20-v1000-c20000-S107009952-009-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3386438-1305586142/watcher-3386438-1305586142 -o /tmp/evaluation-result-3386438-1305586142/solver-3386438-1305586142 -C 5000 -W 5100 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3386438-1305586142.cnf -seed 2076694856 -budget 5000

TIME LIMIT= 5000 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 5093bfaf2b5c3f3d3bcb50ce84a28c32
RANDOM SEED=2076694856

node149.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      14718472 kB
Buffers:         91276 kB
Cached:         617140 kB
SwapCached:      36240 kB
Active:        8764412 kB
Inactive:      9188124 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      14718472 kB
SwapTotal:    67111528 kB
SwapFree:     45358800 kB
Dirty:           55884 kB
Writeback:           0 kB
AnonPages:    17220584 kB
Mapped:          26712 kB
Slab:            98504 kB
PageTables:     112528 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 156255680 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73468 MiB
End job on node149 at 2011-05-17 01:49:41