Trace number 3370906

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.43 1203.71

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m52-q1041.25-n50-p30-s30907550.cnf
MD5SUM002a4cb53ea02f66e0c3418dc8834f7c
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark2.82957
Satisfiable
(Un)Satisfiability was proved
Number of variables2561
Number of clauses490309
Sum of the clauses size983079
Maximum clause length52
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2490259
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 550

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3370906-1305432966/watcher-3370906-1305432966 -o /tmp/evaluation-result-3370906-1305432966/solver-3370906-1305432966 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3370906-1305432966.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.44 4.06 5.15 5/351 24612
/proc/meminfo: memFree=25764792/32951124 swapFree=40554400/67111528
[pid=24612] ppid=24610 vsize=10724 CPUtime=0 cores=1,3,5,7
/proc/24612/stat : 24612 (plingeling) R 24610 24612 17363 0 -1 4202496 773 1255 0 0 0 0 0 0 21 0 1 0 654897088 10981376 703 33554432000 4194304 4417452 140733598173104 18446744073709551615 230809165568 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/24612/statm: 2681 703 147 55 0 567 0

[startup+0.0451931 s]
/proc/loadavg: 3.44 4.06 5.15 5/351 24612
/proc/meminfo: memFree=25764792/32951124 swapFree=40554400/67111528
[pid=24612] ppid=24610 vsize=14236 CPUtime=0.04 cores=1,3,5,7
/proc/24612/stat : 24612 (plingeling) R 24610 24612 17363 0 -1 4202496 1736 1255 0 0 4 0 0 0 21 0 1 0 654897088 14577664 1582 33554432000 4194304 4417452 140733598173104 18446744073709551615 4231321 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/24612/statm: 3559 1582 148 55 0 1445 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 14236

[startup+0.100173 s]
/proc/loadavg: 3.44 4.06 5.15 5/351 24612
/proc/meminfo: memFree=25764792/32951124 swapFree=40554400/67111528
[pid=24612] ppid=24610 vsize=26524 CPUtime=0.09 cores=1,3,5,7
/proc/24612/stat : 24612 (plingeling) R 24610 24612 17363 0 -1 4202496 4808 1255 0 0 9 0 0 0 21 0 1 0 654897088 27160576 4654 33554432000 4194304 4417452 140733598173104 18446744073709551615 230809165568 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/24612/statm: 6631 4654 148 55 0 4517 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 26524

[startup+0.301138 s]
/proc/loadavg: 3.44 4.06 5.15 5/351 24612
/proc/meminfo: memFree=25764792/32951124 swapFree=40554400/67111528
[pid=24612] ppid=24610 vsize=75676 CPUtime=0.29 cores=1,3,5,7
/proc/24612/stat : 24612 (plingeling) R 24610 24612 17363 0 -1 4202496 17096 1255 0 0 28 1 0 0 22 0 1 0 654897088 77492224 16942 33554432000 4194304 4417452 140733598173104 18446744073709551615 230809165568 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/24612/statm: 18919 16942 148 55 0 16805 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 75676

[startup+0.70103 s]
/proc/loadavg: 3.44 4.06 5.15 5/351 24612
/proc/meminfo: memFree=25764792/32951124 swapFree=40554400/67111528
[pid=24612] ppid=24610 vsize=120520 CPUtime=1.82 cores=1,3,5,7
/proc/24612/stat : 24612 (plingeling) S 24610 24612 17363 0 -1 4202496 19952 1255 0 0 180 2 0 0 22 0 5 0 654897088 123412480 17944 33554432000 4194304 4417452 140733598173104 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/24612/statm: 30130 17944 175 55 0 28016 0
Current children cumulated CPU time (s) 1.82
Current children cumulated vsize (KiB) 120520

[startup+1.50089 s]
/proc/loadavg: 3.44 4.06 5.15 9/356 24622
/proc/meminfo: memFree=25675456/32951124 swapFree=40554400/67111528
[pid=24612] ppid=24610 vsize=201292 CPUtime=5 cores=1,3,5,7
/proc/24612/stat : 24612 (plingeling) S 24610 24612 17363 0 -1 4202496 40346 1255 0 0 496 4 0 0 22 0 5 0 654897088 206123008 21985 33554432000 4194304 4417452 140733598173104 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/24612/statm: 50323 21985 180 55 0 48209 0
[pid=24612/tid=24619] ppid=24610 vsize=201292 CPUtime=1.16 cores=1,3,5,7
/proc/24612/task/24619/stat : 24619 (plingeling) R 24610 24612 17363 0 -1 4202560 5290 1255 0 0 116 0 0 0 25 0 5 0 654897120 206123008 21985 33554432000 4194304 4417452 140733598173104 18446744073709551615 4249130 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24612/tid=24620] ppid=24610 vsize=201292 CPUtime=1.16 cores=1,3,5,7
/proc/24612/task/24620/stat : 24620 (plingeling) R 24610 24612 17363 0 -1 4202560 4868 1255 0 0 116 0 0 0 25 0 5 0 654897120 206123008 21985 33554432000 4194304 4417452 140733598173104 18446744073709551615 4249597 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24612/tid=24621] ppid=24610 vsize=201292 CPUtime=1.15 cores=1,3,5,7
/proc/24612/task/24621/stat : 24621 (plingeling) R 24610 24612 17363 0 -1 4202560 5514 1255 0 0 115 0 0 0 25 0 5 0 654897120 206123008 21985 33554432000 4194304 4417452 140733598173104 18446744073709551615 4260503 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=24612/tid=24622] ppid=24610 vsize=201292 CPUtime=1.17 cores=1,3,5,7
/proc/24612/task/24622/stat : 24622 (plingeling) R 24610 24612 17363 0 -1 4202560 7566 1255 0 0 117 0 0 0 25 0 5 0 654897120 206123008 21985 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 5
Current children cumulated vsize (KiB) 201292

[startup+3.10059 s]
/proc/loadavg: 3.44 4.06 5.15 9/356 24622
/proc/meminfo: memFree=25654988/32951124 swapFree=40554400/67111528
[pid=24612] ppid=24610 vsize=228436 CPUtime=11.39 cores=1,3,5,7
/proc/24612/stat : 24612 (plingeling) S 24610 24612 17363 0 -1 4202496 50387 1255 0 0 1133 6 0 0 22 0 5 0 654897088 233918464 28773 33554432000 4194304 4417452 140733598173104 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/24612/statm: 57109 28773 180 55 0 54995 0
[pid=24612/tid=24619] ppid=24610 vsize=228436 CPUtime=2.77 cores=1,3,5,7
/proc/24612/task/24619/stat : 24619 (plingeling) R 24610 24612 17363 0 -1 4202560 5942 1255 0 0 276 1 0 0 25 0 5 0 654897120 233918464 28773 33554432000 4194304 4417452 140733598173104 18446744073709551615 4258472 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24612/tid=24620] ppid=24610 vsize=228436 CPUtime=2.76 cores=1,3,5,7
/proc/24612/task/24620/stat : 24620 (plingeling) R 24610 24612 17363 0 -1 4202560 5523 1255 0 0 276 0 0 0 25 0 5 0 654897120 233918464 28773 33554432000 4194304 4417452 140733598173104 18446744073709551615 4235968 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24612/tid=24621] ppid=24610 vsize=228436 CPUtime=2.74 cores=1,3,5,7
/proc/24612/task/24621/stat : 24621 (plingeling) R 24610 24612 17363 0 -1 4202560 6167 1255 0 0 274 0 0 0 25 0 5 0 654897120 233918464 28773 33554432000 4194304 4417452 140733598173104 18446744073709551615 4258587 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=24612/tid=24622] ppid=24610 vsize=228436 CPUtime=2.76 cores=1,3,5,7
/proc/24612/task/24622/stat : 24622 (plingeling) R 24610 24612 17363 0 -1 4202560 15647 1255 0 0 275 1 0 0 25 0 5 0 654897120 233918464 28773 33554432000 4194304 4417452 140733598173104 18446744073709551615 4250502 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 11.39
Current children cumulated vsize (KiB) 228436

[startup+6.301 s]
/proc/loadavg: 3.80 4.12 5.16 10/356 24623
/proc/meminfo: memFree=25605420/32951124 swapFree=40554400/67111528
[pid=24612] ppid=24610 vsize=244080 CPUtime=24.15 cores=1,3,5,7
/proc/24612/stat : 24612 (plingeling) S 24610 24612 17363 0 -1 4202496 63719 1255 0 0 2408 7 0 0 22 0 5 0 654897088 249937920 32684 33554432000 4194304 4417452 140733598173104 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/24612/statm: 61020 32684 180 55 0 58906 0
[pid=24612/tid=24619] ppid=24610 vsize=244080 CPUtime=5.97 cores=1,3,5,7
/proc/24612/task/24619/stat : 24619 (plingeling) R 24610 24612 17363 0 -1 4202560 7248 1255 0 0 596 1 0 0 25 0 5 0 654897120 249937920 32684 33554432000 4194304 4417452 140733598173104 18446744073709551615 4236280 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24612/tid=24620] ppid=24610 vsize=244080 CPUtime=5.96 cores=1,3,5,7
/proc/24612/task/24620/stat : 24620 (plingeling) R 24610 24612 17363 0 -1 4202560 6828 1255 0 0 596 0 0 0 25 0 5 0 654897120 249937920 32684 33554432000 4194304 4417452 140733598173104 18446744073709551615 4248238 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24612/tid=24621] ppid=24610 vsize=244080 CPUtime=5.9 cores=1,3,5,7
/proc/24612/task/24621/stat : 24621 (plingeling) R 24610 24612 17363 0 -1 4202560 6167 1255 0 0 590 0 0 0 25 0 5 0 654897120 249937920 32684 33554432000 4194304 4417452 140733598173104 18446744073709551615 4376559 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=24612/tid=24622] ppid=24610 vsize=244080 CPUtime=5.97 cores=1,3,5,7
/proc/24612/task/24622/stat : 24622 (plingeling) R 24610 24612 17363 0 -1 4202560 26368 1255 0 0 595 2 0 0 25 0 5 0 654897120 249937920 32684 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364972 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 24.15
Current children cumulated vsize (KiB) 244080

[startup+12.7008 s]
/proc/loadavg: 4.14 4.19 5.18 10/356 24623
/proc/meminfo: memFree=25530816/32951124 swapFree=40554400/67111528
[pid=24612] ppid=24610 vsize=282116 CPUtime=49.68 cores=1,3,5,7
/proc/24612/stat : 24612 (plingeling) S 24610 24612 17363 0 -1 4202496 84209 1255 0 0 4959 9 0 0 22 0 5 0 654897088 288886784 42193 33554432000 4194304 4417452 140733598173104 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/24612/statm: 70529 42193 180 55 0 68415 0
[pid=24612/tid=24619] ppid=24610 vsize=282116 CPUtime=12.37 cores=1,3,5,7
/proc/24612/task/24619/stat : 24619 (plingeling) R 24610 24612 17363 0 -1 4202560 7248 1255 0 0 1236 1 0 0 25 0 5 0 654897120 288886784 42193 33554432000 4194304 4417452 140733598173104 18446744073709551615 4236319 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24612/tid=24620] ppid=24610 vsize=282116 CPUtime=12.36 cores=1,3,5,7
/proc/24612/task/24620/stat : 24620 (plingeling) R 24610 24612 17363 0 -1 4202560 6828 1255 0 0 1236 0 0 0 25 0 5 0 654897120 288886784 42193 33554432000 4194304 4417452 140733598173104 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24612/tid=24621] ppid=24610 vsize=282116 CPUtime=12.23 cores=1,3,5,7
/proc/24612/task/24621/stat : 24621 (plingeling) R 24610 24612 17363 0 -1 4202560 8215 1255 0 0 1223 0 0 0 25 0 5 0 654897120 288886784 42193 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=24612/tid=24622] ppid=24610 vsize=282116 CPUtime=12.37 cores=1,3,5,7
/proc/24612/task/24622/stat : 24622 (plingeling) R 24610 24612 17363 0 -1 4202560 44810 1255 0 0 1233 4 0 0 25 0 5 0 654897120 288886784 42193 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 49.68
Current children cumulated vsize (KiB) 282116

[startup+25.5005 s]
/proc/loadavg: 4.99 4.37 5.22 9/356 24659
/proc/meminfo: memFree=25277372/32951124 swapFree=40554400/67111528
[pid=24612] ppid=24610 vsize=339316 CPUtime=100.74 cores=1,3,5,7
/proc/24612/stat : 24612 (plingeling) S 24610 24612 17363 0 -1 4202496 138850 1255 0 0 10057 17 0 0 22 0 5 0 654897088 347459584 54528 33554432000 4194304 4417452 140733598173104 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/24612/statm: 84829 54555 180 55 0 82715 0
[pid=24612/tid=24619] ppid=24610 vsize=339316 CPUtime=25.17 cores=1,3,5,7
/proc/24612/task/24619/stat : 24619 (plingeling) R 24610 24612 17363 0 -1 4202560 9286 1255 0 0 2516 1 0 0 25 0 5 0 654897120 347459584 54568 33554432000 4194304 4417452 140733598173104 18446744073709551615 4236427 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24612/tid=24620] ppid=24610 vsize=339316 CPUtime=25.16 cores=1,3,5,7
/proc/24612/task/24620/stat : 24620 (plingeling) R 24610 24612 17363 0 -1 4202560 6828 1255 0 0 2516 0 0 0 25 0 5 0 654897120 347459584 54584 33554432000 4194304 4417452 140733598173104 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24612/tid=24621] ppid=24610 vsize=339316 CPUtime=24.89 cores=1,3,5,7
/proc/24612/task/24621/stat : 24621 (plingeling) R 24610 24612 17363 0 -1 4202560 11205 1255 0 0 2488 1 0 0 25 0 5 0 654897120 347459584 54601 33554432000 4194304 4417452 140733598173104 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=24612/tid=24622] ppid=24610 vsize=339316 CPUtime=25.17 cores=1,3,5,7
/proc/24612/task/24622/stat : 24622 (plingeling) R 24610 24612 17363 0 -1 4202560 94511 1255 0 0 2505 12 0 0 25 0 5 0 654897120 347459584 54616 33554432000 4194304 4417452 140733598173104 18446744073709551615 230808859741 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 100.74
Current children cumulated vsize (KiB) 339316

[startup+51.1008 s]
/proc/loadavg: 6.09 4.68 5.30 9/356 24660
/proc/meminfo: memFree=24902096/32951124 swapFree=40554400/67111528
[pid=24612] ppid=24610 vsize=493892 CPUtime=202.87 cores=1,3,5,7
/proc/24612/stat : 24612 (plingeling) S 24610 24612 17363 0 -1 4202496 217682 1255 0 0 20260 27 0 0 22 0 5 0 654897088 505745408 95165 33554432000 4194304 4417452 140733598173104 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/24612/statm: 123473 95165 180 55 0 121359 0
[pid=24612/tid=24619] ppid=24610 vsize=493892 CPUtime=50.76 cores=1,3,5,7
/proc/24612/task/24619/stat : 24619 (plingeling) R 24610 24612 17363 0 -1 4202560 14097 1255 0 0 5075 1 0 0 25 0 5 0 654897120 505745408 95165 33554432000 4194304 4417452 140733598173104 18446744073709551615 4321296 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24612/tid=24620] ppid=24610 vsize=493892 CPUtime=50.76 cores=1,3,5,7
/proc/24612/task/24620/stat : 24620 (plingeling) R 24610 24612 17363 0 -1 4202560 6930 1255 0 0 5076 0 0 0 25 0 5 0 654897120 505745408 95165 33554432000 4194304 4417452 140733598173104 18446744073709551615 4366646 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24612/tid=24621] ppid=24610 vsize=493892 CPUtime=50.21 cores=1,3,5,7
/proc/24612/task/24621/stat : 24621 (plingeling) R 24610 24612 17363 0 -1 4202560 16642 1255 0 0 5020 1 0 0 25 0 5 0 654897120 505745408 95165 33554432000 4194304 4417452 140733598173104 18446744073709551615 4258267 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=24612/tid=24622] ppid=24610 vsize=493892 CPUtime=50.77 cores=1,3,5,7
/proc/24612/task/24622/stat : 24622 (plingeling) R 24610 24612 17363 0 -1 4202560 162905 1255 0 0 5057 20 0 0 25 0 5 0 654897120 505745408 95165 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 202.87
Current children cumulated vsize (KiB) 493892

[startup+102.313 s]
/proc/loadavg: 7.29 5.22 5.45 9/356 24661
/proc/meminfo: memFree=24666640/32951124 swapFree=40554400/67111528
[pid=24612] ppid=24610 vsize=647552 CPUtime=407.16 cores=1,3,5,7
/proc/24612/stat : 24612 (plingeling) S 24610 24612 17363 0 -1 4202496 301740 1255 0 0 40676 40 0 0 22 0 5 0 654897088 663093248 133580 33554432000 4194304 4417452 140733598173104 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/24612/statm: 161888 133580 180 55 0 159774 0
[pid=24612/tid=24619] ppid=24610 vsize=647552 CPUtime=101.98 cores=1,3,5,7
/proc/24612/task/24619/stat : 24619 (plingeling) R 24610 24612 17363 0 -1 4202560 21652 1255 0 0 10196 2 0 0 25 0 5 0 654897120 663093248 133580 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24612/tid=24620] ppid=24610 vsize=647552 CPUtime=101.98 cores=1,3,5,7
/proc/24612/task/24620/stat : 24620 (plingeling) R 24610 24612 17363 0 -1 4202560 7800 1255 0 0 10197 1 0 0 25 0 5 0 654897120 663093248 133580 33554432000 4194304 4417452 140733598173104 18446744073709551615 4236017 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24612/tid=24621] ppid=24610 vsize=647552 CPUtime=100.86 cores=1,3,5,7
/proc/24612/task/24621/stat : 24621 (plingeling) R 24610 24612 17363 0 -1 4202560 17086 1255 0 0 10085 1 0 0 25 0 5 0 654897120 663093248 133580 33554432000 4194304 4417452 140733598173104 18446744073709551615 4235378 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=24612/tid=24622] ppid=24610 vsize=647552 CPUtime=101.98 cores=1,3,5,7
/proc/24612/task/24622/stat : 24622 (plingeling) R 24610 24612 17363 0 -1 4202560 238094 1255 0 0 10166 32 0 0 25 0 5 0 654897120 663093248 133580 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 407.16
Current children cumulated vsize (KiB) 647552

[startup+162.301 s]
/proc/loadavg: 7.81 5.74 5.61 9/356 24663
/proc/meminfo: memFree=24298828/32951124 swapFree=40554400/67111528
[pid=24612] ppid=24610 vsize=760668 CPUtime=646.46 cores=1,3,5,7
/proc/24612/stat : 24612 (plingeling) S 24610 24612 17363 0 -1 4202496 431490 1255 0 0 64588 58 0 0 22 0 5 0 654897088 778924032 161859 33554432000 4194304 4417452 140733598173104 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/24612/statm: 190167 161859 180 55 0 188053 0
[pid=24612/tid=24619] ppid=24610 vsize=760668 CPUtime=161.97 cores=1,3,5,7
/proc/24612/task/24619/stat : 24619 (plingeling) R 24610 24612 17363 0 -1 4202560 21652 1255 0 0 16195 2 0 0 25 0 5 0 654897120 778924032 161859 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24612/tid=24620] ppid=24610 vsize=760668 CPUtime=161.96 cores=1,3,5,7
/proc/24612/task/24620/stat : 24620 (plingeling) R 24610 24612 17363 0 -1 4202560 14474 1255 0 0 16195 1 0 0 25 0 5 0 654897120 778924032 161859 33554432000 4194304 4417452 140733598173104 18446744073709551615 4260667 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24612/tid=24621] ppid=24610 vsize=760668 CPUtime=160.2 cores=1,3,5,7
/proc/24612/task/24621/stat : 24621 (plingeling) R 24610 24612 17363 0 -1 4202560 18421 1255 0 0 16019 1 0 0 25 0 5 0 654897120 778924032 161859 33554432000 4194304 4417452 140733598173104 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=24612/tid=24622] ppid=24610 vsize=760668 CPUtime=161.97 cores=1,3,5,7
/proc/24612/task/24622/stat : 24622 (plingeling) R 24610 24612 17363 0 -1 4202560 359835 1255 0 0 16148 49 0 0 25 0 5 0 654897120 778924032 161859 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 646.46
Current children cumulated vsize (KiB) 760668


################
# More data... #
################

[pid=24612/tid=24620] ppid=24610 vsize=2480780 CPUtime=701.98 cores=1,3,5,7
/proc/24612/task/24620/stat : 24620 (plingeling) R 24610 24612 17363 0 -1 4202560 224910 1255 0 0 70166 32 0 0 25 0 5 0 654897120 2540318720 591887 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24612/tid=24621] ppid=24610 vsize=2480780 CPUtime=694.29 cores=1,3,5,7
/proc/24612/task/24621/stat : 24621 (plingeling) R 24610 24612 17363 0 -1 4202560 240857 1255 0 0 69398 31 0 0 25 0 5 0 654897120 2540318720 591887 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=24612/tid=24622] ppid=24610 vsize=2480780 CPUtime=701.99 cores=1,3,5,7
/proc/24612/task/24622/stat : 24622 (plingeling) R 24610 24612 17363 0 -1 4202560 881587 1255 0 0 70072 127 0 0 25 0 5 0 654897120 2540318720 591887 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2800.59
Current children cumulated vsize (KiB) 2480780

[startup+762.313 s]
/proc/loadavg: 8.06 7.73 6.75 9/356 24679
/proc/meminfo: memFree=21043260/32951124 swapFree=40554400/67111528
[pid=24612] ppid=24610 vsize=2706516 CPUtime=3039.97 cores=1,3,5,7
/proc/24612/stat : 24612 (plingeling) S 24610 24612 17363 0 -1 4202496 1816296 1255 0 0 303741 256 0 0 22 0 5 0 654897088 2771472384 648321 33554432000 4194304 4417452 140733598173104 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/24612/statm: 676629 648321 180 55 0 674515 0
[pid=24612/tid=24619] ppid=24610 vsize=2706516 CPUtime=762 cores=1,3,5,7
/proc/24612/task/24619/stat : 24619 (plingeling) R 24610 24612 17363 0 -1 4202560 281616 1255 0 0 76163 37 0 0 25 0 5 0 654897120 2771472384 648321 33554432000 4194304 4417452 140733598173104 18446744073709551615 4366469 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24612/tid=24620] ppid=24610 vsize=2706516 CPUtime=761.99 cores=1,3,5,7
/proc/24612/task/24620/stat : 24620 (plingeling) R 24610 24612 17363 0 -1 4202560 298021 1255 0 0 76159 40 0 0 25 0 5 0 654897120 2771472384 648321 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24612/tid=24621] ppid=24610 vsize=2706516 CPUtime=753.62 cores=1,3,5,7
/proc/24612/task/24621/stat : 24621 (plingeling) R 24610 24612 17363 0 -1 4202560 267564 1255 0 0 75328 34 0 0 25 0 5 0 654897120 2771472384 648321 33554432000 4194304 4417452 140733598173104 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=24612/tid=24622] ppid=24610 vsize=2706516 CPUtime=762 cores=1,3,5,7
/proc/24612/task/24622/stat : 24622 (plingeling) R 24610 24612 17363 0 -1 4202560 951987 1255 0 0 76059 141 0 0 25 0 5 0 654897120 2771472384 648321 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3039.97
Current children cumulated vsize (KiB) 2706516

[startup+822.301 s]
/proc/loadavg: 8.06 7.79 6.83 9/356 24680
/proc/meminfo: memFree=20710916/32951124 swapFree=40554400/67111528
[pid=24612] ppid=24610 vsize=2861836 CPUtime=3279.27 cores=1,3,5,7
/proc/24612/stat : 24612 (plingeling) S 24610 24612 17363 0 -1 4202496 1904278 1255 0 0 327661 266 0 0 22 0 5 0 654897088 2930520064 687151 33554432000 4194304 4417452 140733598173104 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/24612/statm: 715459 687151 180 55 0 713345 0
[pid=24612/tid=24619] ppid=24610 vsize=2861836 CPUtime=821.99 cores=1,3,5,7
/proc/24612/task/24619/stat : 24619 (plingeling) R 24610 24612 17363 0 -1 4202560 285989 1255 0 0 82161 38 0 0 25 0 5 0 654897120 2930520064 687151 33554432000 4194304 4417452 140733598173104 18446744073709551615 4307546 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24612/tid=24620] ppid=24610 vsize=2861836 CPUtime=821.99 cores=1,3,5,7
/proc/24612/task/24620/stat : 24620 (plingeling) R 24610 24612 17363 0 -1 4202560 370993 1255 0 0 82151 48 0 0 25 0 5 0 654897120 2930520064 687151 33554432000 4194304 4417452 140733598173104 18446744073709551615 4251160 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24612/tid=24621] ppid=24610 vsize=2861836 CPUtime=812.96 cores=1,3,5,7
/proc/24612/task/24621/stat : 24621 (plingeling) R 24610 24612 17363 0 -1 4202560 278201 1255 0 0 81260 36 0 0 25 0 5 0 654897120 2930520064 687151 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=24612/tid=24622] ppid=24610 vsize=2861836 CPUtime=821.99 cores=1,3,5,7
/proc/24612/task/24622/stat : 24622 (plingeling) R 24610 24612 17363 0 -1 4202560 951987 1255 0 0 82058 141 0 0 25 0 5 0 654897120 2930520064 687151 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3279.27
Current children cumulated vsize (KiB) 2861836

[startup+882.301 s]
/proc/loadavg: 8.05 7.84 6.91 9/356 24682
/proc/meminfo: memFree=20920340/32951124 swapFree=40554400/67111528
[pid=24612] ppid=24610 vsize=3001276 CPUtime=3518.61 cores=1,3,5,7
/proc/24612/stat : 24612 (plingeling) S 24610 24612 17363 0 -1 4202496 1987246 1255 0 0 351583 278 0 0 22 0 5 0 654897088 3073306624 722011 33554432000 4194304 4417452 140733598173104 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/24612/statm: 750319 722011 180 55 0 748205 0
[pid=24612/tid=24619] ppid=24610 vsize=3001276 CPUtime=881.99 cores=1,3,5,7
/proc/24612/task/24619/stat : 24619 (plingeling) R 24610 24612 17363 0 -1 4202560 331940 1255 0 0 88153 46 0 0 25 0 5 0 654897120 3073306624 722011 33554432000 4194304 4417452 140733598173104 18446744073709551615 4263050 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24612/tid=24620] ppid=24610 vsize=3001276 CPUtime=881.99 cores=1,3,5,7
/proc/24612/task/24620/stat : 24620 (plingeling) R 24610 24612 17363 0 -1 4202560 399818 1255 0 0 88148 51 0 0 25 0 5 0 654897120 3073306624 722011 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24612/tid=24621] ppid=24610 vsize=3001276 CPUtime=872.29 cores=1,3,5,7
/proc/24612/task/24621/stat : 24621 (plingeling) R 24610 24612 17363 0 -1 4202560 286393 1255 0 0 87192 37 0 0 25 0 5 0 654897120 3073306624 722011 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=24612/tid=24622] ppid=24610 vsize=3001276 CPUtime=881.99 cores=1,3,5,7
/proc/24612/task/24622/stat : 24622 (plingeling) R 24610 24612 17363 0 -1 4202560 951987 1255 0 0 88058 141 0 0 25 0 5 0 654897120 3073306624 722011 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364984 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3518.61
Current children cumulated vsize (KiB) 3001276

[startup+942.301 s]
/proc/loadavg: 8.02 7.87 6.97 9/356 24684
/proc/meminfo: memFree=20883612/32951124 swapFree=40554400/67111528
[pid=24612] ppid=24610 vsize=3160460 CPUtime=3757.95 cores=1,3,5,7
/proc/24612/stat : 24612 (plingeling) S 24610 24612 17363 0 -1 4202496 2048998 1255 0 0 375509 286 0 0 22 0 5 0 654897088 3236311040 761807 33554432000 4194304 4417452 140733598173104 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/24612/statm: 790115 761807 180 55 0 788001 0
[pid=24612/tid=24619] ppid=24610 vsize=3160460 CPUtime=941.99 cores=1,3,5,7
/proc/24612/task/24619/stat : 24619 (plingeling) R 24610 24612 17363 0 -1 4202560 377308 1255 0 0 94148 51 0 0 25 0 5 0 654897120 3236311040 761807 33554432000 4194304 4417452 140733598173104 18446744073709551615 4263080 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24612/tid=24620] ppid=24610 vsize=3160460 CPUtime=941.99 cores=1,3,5,7
/proc/24612/task/24620/stat : 24620 (plingeling) R 24610 24612 17363 0 -1 4202560 399818 1255 0 0 94148 51 0 0 25 0 5 0 654897120 3236311040 761807 33554432000 4194304 4417452 140733598173104 18446744073709551615 4309920 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24612/tid=24621] ppid=24610 vsize=3160460 CPUtime=931.63 cores=1,3,5,7
/proc/24612/task/24621/stat : 24621 (plingeling) R 24610 24612 17363 0 -1 4202560 302777 1255 0 0 93123 40 0 0 25 0 5 0 654897120 3236311040 761807 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=24612/tid=24622] ppid=24610 vsize=3160460 CPUtime=941.99 cores=1,3,5,7
/proc/24612/task/24622/stat : 24622 (plingeling) R 24610 24612 17363 0 -1 4202560 951987 1255 0 0 94058 141 0 0 25 0 5 0 654897120 3236311040 761807 33554432000 4194304 4417452 140733598173104 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3757.95
Current children cumulated vsize (KiB) 3160460

[startup+1002.3 s]
/proc/loadavg: 8.00 7.89 7.03 9/356 24685
/proc/meminfo: memFree=21392008/32951124 swapFree=40554400/67111528
[pid=24612] ppid=24610 vsize=2601272 CPUtime=3997.29 cores=1,3,5,7
/proc/24612/stat : 24612 (plingeling) S 24610 24612 17363 0 -1 4202496 2209102 1255 0 0 399418 311 0 0 22 0 5 0 654897088 2663702528 622010 33554432000 4194304 4417452 140733598173104 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/24612/statm: 650318 622010 180 55 0 648204 0
[pid=24612/tid=24619] ppid=24610 vsize=2601272 CPUtime=1001.99 cores=1,3,5,7
/proc/24612/task/24619/stat : 24619 (plingeling) R 24610 24612 17363 0 -1 4202560 394786 1255 0 0 100146 53 0 0 25 0 5 0 654897120 2663702528 622010 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24612/tid=24620] ppid=24610 vsize=2601272 CPUtime=1001.98 cores=1,3,5,7
/proc/24612/task/24620/stat : 24620 (plingeling) R 24610 24612 17363 0 -1 4202560 480734 1255 0 0 100134 64 0 0 25 0 5 0 654897120 2663702528 622010 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24612/tid=24621] ppid=24610 vsize=2601272 CPUtime=990.97 cores=1,3,5,7
/proc/24612/task/24621/stat : 24621 (plingeling) R 24610 24612 17363 0 -1 4202560 302777 1255 0 0 99057 40 0 0 25 0 5 0 654897120 2663702528 622010 33554432000 4194304 4417452 140733598173104 18446744073709551615 4311298 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=24612/tid=24622] ppid=24610 vsize=2601272 CPUtime=1001.99 cores=1,3,5,7
/proc/24612/task/24622/stat : 24622 (plingeling) R 24610 24612 17363 0 -1 4202560 1013697 1255 0 0 100049 150 0 0 25 0 5 0 654897120 2663702528 622010 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3997.29
Current children cumulated vsize (KiB) 2601272

[startup+1062.3 s]
/proc/loadavg: 8.00 7.91 7.09 9/356 24687
/proc/meminfo: memFree=21472460/32951124 swapFree=40554400/67111528
[pid=24612] ppid=24610 vsize=2441384 CPUtime=4236.65 cores=1,3,5,7
/proc/24612/stat : 24612 (plingeling) S 24610 24612 17363 0 -1 4202496 2300300 1255 0 0 423339 326 0 0 22 0 5 0 654897088 2499977216 582038 33554432000 4194304 4417452 140733598173104 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/24612/statm: 610346 582038 180 55 0 608232 0
[pid=24612/tid=24619] ppid=24610 vsize=2441384 CPUtime=1061.99 cores=1,3,5,7
/proc/24612/task/24619/stat : 24619 (plingeling) R 24610 24612 17363 0 -1 4202560 401729 1255 0 0 106144 55 0 0 25 0 5 0 654897120 2499977216 582038 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24612/tid=24620] ppid=24610 vsize=2441384 CPUtime=1061.99 cores=1,3,5,7
/proc/24612/task/24620/stat : 24620 (plingeling) R 24610 24612 17363 0 -1 4202560 482940 1255 0 0 106133 66 0 0 25 0 5 0 654897120 2499977216 582038 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24612/tid=24621] ppid=24610 vsize=2441384 CPUtime=1050.32 cores=1,3,5,7
/proc/24612/task/24621/stat : 24621 (plingeling) R 24610 24612 17363 0 -1 4202560 353429 1255 0 0 104984 48 0 0 25 0 5 0 654897120 2499977216 582038 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=24612/tid=24622] ppid=24610 vsize=2441384 CPUtime=1062 cores=1,3,5,7
/proc/24612/task/24622/stat : 24622 (plingeling) R 24610 24612 17363 0 -1 4202560 1045094 1255 0 0 106047 153 0 0 25 0 5 0 654897120 2499977216 582038 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364693 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4236.65
Current children cumulated vsize (KiB) 2441384

[startup+1122.3 s]
/proc/loadavg: 8.08 7.94 7.15 9/356 24688
/proc/meminfo: memFree=21011408/32951124 swapFree=40554400/67111528
[pid=24612] ppid=24610 vsize=2664488 CPUtime=4475.98 cores=1,3,5,7
/proc/24612/stat : 24612 (plingeling) S 24610 24612 17363 0 -1 4202496 2486520 1255 0 0 447243 355 0 0 22 0 5 0 654897088 2728435712 637814 33554432000 4194304 4417452 140733598173104 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/24612/statm: 666122 637814 180 55 0 664008 0
[pid=24612/tid=24619] ppid=24610 vsize=2664488 CPUtime=1121.99 cores=1,3,5,7
/proc/24612/task/24619/stat : 24619 (plingeling) R 24610 24612 17363 0 -1 4202560 405748 1255 0 0 112143 56 0 0 25 0 5 0 654897120 2728435712 637814 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24612/tid=24620] ppid=24610 vsize=2664488 CPUtime=1121.99 cores=1,3,5,7
/proc/24612/task/24620/stat : 24620 (plingeling) R 24610 24612 17363 0 -1 4202560 536086 1255 0 0 112127 72 0 0 25 0 5 0 654897120 2728435712 637814 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24612/tid=24621] ppid=24610 vsize=2664488 CPUtime=1109.63 cores=1,3,5,7
/proc/24612/task/24621/stat : 24621 (plingeling) R 24610 24612 17363 0 -1 4202560 403741 1255 0 0 110906 57 0 0 25 0 5 0 654897120 2728435712 637814 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=24612/tid=24622] ppid=24610 vsize=2664488 CPUtime=1122 cores=1,3,5,7
/proc/24612/task/24622/stat : 24622 (plingeling) R 24610 24612 17363 0 -1 4202560 1123837 1255 0 0 112034 166 0 0 25 0 5 0 654897120 2728435712 637814 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4475.98
Current children cumulated vsize (KiB) 2664488

[startup+1182.3 s]
/proc/loadavg: 8.16 7.98 7.21 9/356 24690
/proc/meminfo: memFree=20700264/32951124 swapFree=40554400/67111528
[pid=24612] ppid=24610 vsize=2801916 CPUtime=4715.31 cores=1,3,5,7
/proc/24612/stat : 24612 (plingeling) S 24610 24612 17363 0 -1 4202496 2610186 1255 0 0 471159 372 0 0 22 0 5 0 654897088 2869161984 672171 33554432000 4194304 4417452 140733598173104 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/24612/statm: 700479 672171 180 55 0 698365 0
[pid=24612/tid=24619] ppid=24610 vsize=2801916 CPUtime=1182 cores=1,3,5,7
/proc/24612/task/24619/stat : 24619 (plingeling) R 24610 24612 17363 0 -1 4202560 454190 1255 0 0 118135 65 0 0 25 0 5 0 654897120 2869161984 672171 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24612/tid=24620] ppid=24610 vsize=2801916 CPUtime=1181.99 cores=1,3,5,7
/proc/24612/task/24620/stat : 24620 (plingeling) R 24610 24612 17363 0 -1 4202560 594926 1255 0 0 118120 79 0 0 25 0 5 0 654897120 2869161984 672171 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24612/tid=24621] ppid=24610 vsize=2801916 CPUtime=1168.97 cores=1,3,5,7
/proc/24612/task/24621/stat : 24621 (plingeling) R 24610 24612 17363 0 -1 4202560 420125 1255 0 0 116838 59 0 0 25 0 5 0 654897120 2869161984 672171 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=24612/tid=24622] ppid=24610 vsize=2801916 CPUtime=1182 cores=1,3,5,7
/proc/24612/task/24622/stat : 24622 (plingeling) R 24610 24612 17363 0 -1 4202560 1123837 1255 0 0 118034 166 0 0 25 0 5 0 654897120 2869161984 672171 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4715.31
Current children cumulated vsize (KiB) 2801916



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.6 s]
/proc/loadavg: 8.18 7.99 7.23 9/356 24691
/proc/meminfo: memFree=20711424/32951124 swapFree=40554400/67111528
[pid=24612] ppid=24610 vsize=2769148 CPUtime=4800.29 cores=1,3,5,7
/proc/24612/stat : 24612 (plingeling) S 24610 24612 17363 0 -1 4202496 2634762 1255 0 0 479654 375 0 0 22 0 5 0 654897088 2835607552 663979 33554432000 4194304 4417452 140733598173104 18446744073709551615 230820968421 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/24612/statm: 692287 663979 180 55 0 690173 0
[pid=24612/tid=24619] ppid=24610 vsize=2769148 CPUtime=1203.31 cores=1,3,5,7
/proc/24612/task/24619/stat : 24619 (plingeling) R 24610 24612 17363 0 -1 4202560 462382 1255 0 0 120265 66 0 0 25 0 5 0 654897120 2835607552 663979 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=24612/tid=24620] ppid=24610 vsize=2769148 CPUtime=1203.31 cores=1,3,5,7
/proc/24612/task/24620/stat : 24620 (plingeling) R 24610 24612 17363 0 -1 4202560 611310 1255 0 0 120250 81 0 0 25 0 5 0 654897120 2835607552 663979 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=24612/tid=24621] ppid=24610 vsize=2769148 CPUtime=1190.02 cores=1,3,5,7
/proc/24612/task/24621/stat : 24621 (plingeling) R 24610 24612 17363 0 -1 4202560 420125 1255 0 0 118943 59 0 0 25 0 5 0 654897120 2835607552 663979 33554432000 4194304 4417452 140733598173104 18446744073709551615 4366130 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=24612/tid=24622] ppid=24610 vsize=2769148 CPUtime=1203.32 cores=1,3,5,7
/proc/24612/task/24622/stat : 24622 (plingeling) R 24610 24612 17363 0 -1 4202560 1123837 1255 0 0 120166 166 0 0 25 0 5 0 654897120 2835607552 663979 33554432000 4194304 4417452 140733598173104 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.29
Current children cumulated vsize (KiB) 2769148

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.71
CPU time (s): 4800.43
CPU user time (s): 4796.58
CPU system time (s): 3.85041
CPU usage (%): 398.803
Max. virtual memory (cumulated for all children) (KiB): 3160460

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4796.58
system time used= 3.85041
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2636018
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 98
involuntary context switches= 33628

runsolver used 3.83042 second user time and 9.46456 second system time

The end

Launcher Data 

Begin job on node104 at 2011-05-15 06:16:06
IDJOB=3370906
IDBENCH=88207
IDSOLVER=1848
FILE ID=node104/3370906-1305432966
RUNJOBID= node104-1305432966-24584
PBS_JOBID= 13324584
Free space on /tmp= 73420 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m52-q1041.25-n50-p30-s30907550.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3370906-1305432966/watcher-3370906-1305432966 -o /tmp/evaluation-result-3370906-1305432966/solver-3370906-1305432966 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3370906-1305432966.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 002a4cb53ea02f66e0c3418dc8834f7c
RANDOM SEED=1707390844

node104.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5293.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25764824 kB
Buffers:        302768 kB
Cached:        2397500 kB
SwapCached:     122128 kB
Active:         959520 kB
Inactive:      5959548 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25764824 kB
SwapTotal:    67111528 kB
SwapFree:     40554400 kB
Dirty:            7104 kB
Writeback:           0 kB
AnonPages:     4110052 kB
Mapped:          11264 kB
Slab:           118060 kB
PageTables:      84504 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 74877156 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73408 MiB
End job on node104 at 2011-05-15 06:36:12