Trace number 1112705

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerNumber of falsified constraintsCPU timeWall clock time
Concrete + CSP4J - MCRW Engine 2008-05-30MSAT (TO)9 3600.01 3624.92

General information on the benchmark

Namecsp/os-gp/
normalized-gp10-05-1070.xml
MD5SUM76853a39367362e3e4e459176be1a8ba
Bench Category2-ARY-INT (binary constraints in intension)
Best result obtained on this benchmarkMOPT
Best Number of falsified constraints1
Best CPU time to get the best result obtained on this benchmark27.8248
Satisfiable
(Un)Satisfiability was proved
Number of variables101
Number of constraints1000
Maximum constraint arity2
Maximum domain size1071
Number of constraints which are defined in extension0
Number of constraints which are defined in intension1000
Global constraints used (with number of constraints)

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data (download as text)

1.77/1.89	c CPU-LOAD 0.93
1.77/1.89	c LOADED 101 V (1071 E), 900 DTPConstraint, 100 DiffConstraint, arity 2, 0.93 cpu
3.25/3.33	s SATISFIABLE
3.25/3.33	o 976
3.25/3.33	v 1070 968 427 1063 141 744 307 247 695 358 1040 661 849 1014 1052 513 359 317 150 22 985 38 701 227 910 1068 397 69 972 198 901 1006 6 433 150 279 42 342 382 985 65 398 1007 355 14 739 750 63 1067 354 133 42 90 19 86 382 248 421 308 508 1000 901 220 598 419 2 1063 842 1009 996 723 385 1058 831 972 265 76 985 610 470 94 264 754 92 104 815 396 1063 94 218 346 1037 572 96 613 505 324 321 23 955 344 
3.25/3.34	o 977
3.25/3.34	v 1070 968 427 1063 141 744 307 247 695 358 1040 661 849 1014 1052 513 359 317 150 22 985 38 58 227 910 1068 397 69 972 198 901 1006 6 433 150 279 42 342 382 985 65 398 1007 355 14 739 750 63 1067 354 133 42 90 19 86 382 248 421 308 508 1000 901 220 598 419 2 1063 842 1009 996 723 385 1058 831 972 265 76 985 610 470 94 264 754 92 104 815 396 1063 94 218 346 1037 572 96 613 505 324 321 23 955 344 
3.25/3.34	o 978
3.25/3.34	v 1070 968 427 1063 141 744 307 247 1000 358 1044 661 849 1014 1052 513 359 317 150 22 985 38 58 239 910 1068 397 69 972 198 901 1006 6 433 150 279 42 342 382 985 65 398 1007 355 14 739 750 63 1067 354 133 42 614 19 86 382 248 421 308 508 1000 901 220 598 419 2 1048 842 1009 996 723 385 1058 831 972 265 76 985 610 470 94 264 754 92 104 815 396 1063 94 218 346 1037 572 96 613 505 324 321 23 955 344 
3.25/3.35	o 979
3.25/3.35	v 1070 968 427 1063 141 744 307 247 1000 358 1044 661 849 1014 1052 513 359 317 150 22 985 38 58 214 910 1069 397 69 972 198 901 1006 6 433 150 279 42 342 382 985 65 398 1007 355 14 739 750 63 1067 354 133 42 614 19 86 382 248 421 308 508 1000 901 220 598 419 2 1048 842 1009 996 723 385 1062 831 972 459 76 985 610 470 94 264 754 92 104 815 396 1063 94 218 346 1037 572 96 613 505 324 321 23 955 344 
3.25/3.35	o 980
3.25/3.35	v 1070 968 427 1054 141 744 307 247 1000 358 1044 661 849 1014 1052 513 359 317 150 22 985 38 58 214 910 1069 397 69 972 198 901 1006 6 433 150 279 42 342 382 985 65 398 1007 355 14 739 750 63 1067 354 133 42 614 19 86 382 248 421 308 508 1000 901 220 920 419 2 1048 842 1009 996 723 385 1062 831 972 459 76 985 610 470 94 264 754 92 104 815 396 1063 94 218 346 1037 572 96 613 505 324 321 23 955 344 
3.25/3.36	o 982
3.25/3.36	v 1070 968 427 1054 141 744 307 247 1000 358 1044 661 849 1014 1052 513 359 317 150 22 985 38 58 214 910 1069 397 69 972 198 901 1006 6 433 150 279 42 342 382 985 65 398 1007 355 14 739 750 63 1067 354 133 42 614 19 86 382 248 421 308 508 1000 901 220 920 419 2 1048 842 1009 996 723 385 1062 831 972 459 76 985 894 470 94 264 754 92 104 815 396 1063 94 218 346 1037 572 96 613 505 324 321 23 955 344 
3.25/3.37	o 983
3.25/3.37	v 1070 968 427 435 141 744 307 247 1000 358 1044 661 849 1014 1068 513 359 317 150 22 985 38 58 214 910 1069 444 69 972 198 901 1006 6 1068 150 279 42 342 382 985 65 399 1007 355 14 739 750 63 1067 354 360 42 614 19 86 382 248 421 308 508 1000 901 220 920 419 2 1048 842 1009 996 723 385 1062 831 972 459 76 985 894 463 94 264 754 92 104 815 396 1065 94 218 346 1039 572 96 613 505 324 321 23 955 344 
3.35/3.45	o 984
3.35/3.45	v 1070 968 586 427 141 744 307 247 1000 358 1059 661 849 1014 1068 513 286 317 339 23 985 38 58 727 910 1065 417 69 972 198 901 1006 6 1065 150 279 42 342 409 985 65 396 1007 355 14 739 756 63 114 354 423 687 93 19 6 382 248 421 25 1069 1000 901 220 920 419 2 1051 842 1009 996 723 385 1056 831 972 459 76 985 894 463 94 264 754 92 104 815 396 1068 94 218 346 1010 422 93 613 505 324 321 542 955 344 
3.35/3.49	o 985
3.35/3.49	v 1070 968 588 422 141 744 307 247 1000 358 1056 53 849 1014 1068 513 286 317 360 681 985 38 58 722 910 1066 438 69 972 198 901 1006 6 407 150 279 42 342 1013 985 65 401 1007 355 14 739 755 63 114 354 433 711 93 19 6 382 248 421 25 1020 1000 901 220 920 419 2 1015 842 1009 996 723 385 1056 831 972 459 76 985 894 463 94 264 754 92 104 815 396 1066 94 218 346 1010 422 93 613 505 324 321 542 955 344 
3.54/3.64	o 986
3.54/3.64	v 1070 968 807 445 141 744 307 247 1000 358 1013 53 622 1014 1053 513 12 317 360 835 985 38 58 249 910 1068 422 69 972 198 901 1006 6 498 150 279 42 342 1037 985 65 440 1007 355 14 739 1070 63 114 354 397 711 256 19 6 382 1011 421 25 1037 1000 901 62 920 284 10 394 842 1009 996 723 385 1060 831 972 459 76 985 894 470 94 264 754 247 104 815 131 1064 94 218 346 1014 422 21 613 505 324 321 542 955 344 
3.54/3.68	o 987
3.54/3.68	v 1070 968 807 440 141 744 307 247 1000 358 1021 57 622 1014 1058 81 12 317 360 835 985 38 58 249 910 1069 422 69 972 198 901 1006 6 386 150 279 42 342 1037 985 65 434 1007 355 14 739 747 63 114 354 362 711 256 19 6 382 1011 421 25 1045 1000 901 59 920 292 481 1052 842 1009 996 723 385 1065 831 972 459 76 985 894 470 94 264 754 247 104 815 131 1066 94 218 346 1054 422 21 613 452 324 321 542 955 344 
3.64/3.75	o 988
3.64/3.75	v 1070 968 807 427 141 744 22 247 1000 358 1067 130 622 1014 1053 33 12 317 360 835 985 38 58 249 910 1068 395 69 972 198 901 1006 6 448 150 279 42 342 1060 985 65 419 1007 355 14 1064 91 63 657 354 367 711 256 19 6 382 915 421 25 1034 1000 901 59 854 292 481 946 842 853 276 723 385 1065 831 972 459 76 985 894 470 94 409 754 247 104 815 58 1062 94 218 346 1011 422 21 613 452 909 321 542 955 344 
3.64/3.79	o 989
3.64/3.79	v 1070 968 807 425 141 744 22 247 1000 358 1002 130 622 1014 1069 33 12 317 360 835 985 38 58 249 910 1068 395 69 972 198 901 1006 6 506 150 279 42 342 1053 985 65 391 1007 355 14 1064 91 63 657 354 369 711 256 19 6 382 915 421 25 1033 1000 901 59 854 292 481 1011 842 853 276 723 385 1062 831 972 459 76 985 894 470 94 328 754 247 104 815 58 1069 94 218 346 1045 422 21 613 452 909 321 542 955 344 
9.09/9.24	o 990
9.09/9.24	v 1070 2 863 1014 9 768 22 20 16 60 194 877 200 52 1053 102 12 39 542 750 28 38 32 93 61 1064 348 203 1059 16 1 170 6 490 263 196 8 41 1065 149 1018 528 810 798 975 1041 39 1065 328 6 1070 192 380 4 6 9 825 601 25 1070 8 7 521 11 722 271 967 2 236 1 61 97 1008 85 4 40 3 151 163 359 712 21 792 866 104 519 19 1070 1051 1049 55 1045 35 199 424 23 30 1 94 163 0 
68.13/68.87	o 991
68.13/68.87	v 1070 68 911 1064 49 841 9 29 1060 90 165 389 761 80 1058 913 17 0 222 616 2 10 2 371 7 62 605 475 22 11 1 21 0 705 149 1008 7 34 1070 38 597 714 40 2 92 9 212 18 506 6 1021 73 452 4 5 22 32 268 995 1060 10 5 124 29 410 623 1060 2 0 1 900 978 1060 20 139 0 3 1012 47 168 640 43 105 17 297 372 16 1015 36 0 3 1000 8 117 617 4 2 1 374 56 0

Verifier Data (download as text)

OK	9	991
9 unsatisfied constraints, 991 satisfied constraints

Watcher Data (download as text)

runsolver version 3.2.5 (c) roussel@cril.univ-artois.fr

command line: /home/evaluation/evaluation/pub/bin/runsolver --timestamp -w /tmp/evaluation-result-1112705-1215872862/watcher-1112705-1215872862 -o /tmp/evaluation-result-1112705-1215872862/solver-1112705-1215872862 -C 3600 -W 4000 -M 900 --output-limit 1,15 HOME/concrete.sh -c -ext 0 -s MCRW -max HOME/instance-1112705-1215872862.xml 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 3600 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 3630 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 4000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 921600 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 972800 KiB
Solver output will be limited to a maximum of 15728640 bytes. The first 1048576 bytes and the last 14680064 bytes will be preserved
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.05 2.00 2.00 4/81 19557
/proc/meminfo: memFree=1834764/2055920 swapFree=4192812/4192956
[pid=19557] ppid=19555 vsize=5356 CPUtime=0
/proc/19557/stat : 19557 (concrete.sh) S 19555 19557 17668 0 -1 4194304 271 0 0 0 0 0 0 0 20 0 1 0 43988726 5484544 230 996147200 4194304 4889804 548682068752 18446744073709551615 272467984466 0 0 4100 65536 18446744071563648864 0 0 17 1 0 0
/proc/19557/statm: 1339 230 191 169 0 50 0
[pid=19558] ppid=19557 vsize=172 CPUtime=0
/proc/19558/stat : 19558 (dirname) R 19557 19557 17668 0 -1 4194304 53 0 0 0 0 0 0 0 22 0 1 0 43988726 176128 28 996147200 4194304 4208292 548682069008 18446744073709551615 272465162147 0 0 4096 0 0 0 0 17 1 0 0
/proc/19558/statm: 43 28 22 3 0 3 0

[startup+0.054516 s]
/proc/loadavg: 2.05 2.00 2.00 4/81 19557
/proc/meminfo: memFree=1834764/2055920 swapFree=4192812/4192956
[pid=19557] ppid=19555 vsize=5356 CPUtime=0
/proc/19557/stat : 19557 (concrete.sh) S 19555 19557 17668 0 -1 4194304 315 128 0 0 0 0 0 0 21 0 1 0 43988726 5484544 242 996147200 4194304 4889804 548682068752 18446744073709551615 272467813188 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/19557/statm: 1339 242 201 169 0 50 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 5356

[startup+0.102519 s]
/proc/loadavg: 2.05 2.00 2.00 4/81 19557
/proc/meminfo: memFree=1834764/2055920 swapFree=4192812/4192956
[pid=19557] ppid=19555 vsize=5356 CPUtime=0
/proc/19557/stat : 19557 (concrete.sh) S 19555 19557 17668 0 -1 4194304 315 128 0 0 0 0 0 0 21 0 1 0 43988726 5484544 242 996147200 4194304 4889804 548682068752 18446744073709551615 272467813188 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/19557/statm: 1339 242 201 169 0 50 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 5356

[startup+0.301531 s]
/proc/loadavg: 2.05 2.00 2.00 4/81 19557
/proc/meminfo: memFree=1834764/2055920 swapFree=4192812/4192956
[pid=19557] ppid=19555 vsize=5356 CPUtime=0
/proc/19557/stat : 19557 (concrete.sh) S 19555 19557 17668 0 -1 4194304 315 128 0 0 0 0 0 0 21 0 1 0 43988726 5484544 242 996147200 4194304 4889804 548682068752 18446744073709551615 272467813188 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/19557/statm: 1339 242 201 169 0 50 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 5356

[startup+0.70256 s]
/proc/loadavg: 2.05 2.00 2.00 4/81 19557
/proc/meminfo: memFree=1834764/2055920 swapFree=4192812/4192956
[pid=19557] ppid=19555 vsize=5356 CPUtime=0
/proc/19557/stat : 19557 (concrete.sh) S 19555 19557 17668 0 -1 4194304 315 128 0 0 0 0 0 0 21 0 1 0 43988726 5484544 242 996147200 4194304 4889804 548682068752 18446744073709551615 272467813188 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/19557/statm: 1339 242 201 169 0 50 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 5356

[startup+1.50962 s]
/proc/loadavg: 2.05 2.00 2.00 4/92 19569
/proc/meminfo: memFree=1817780/2055920 swapFree=4192812/4192956
[pid=19557] ppid=19555 vsize=5356 CPUtime=0
/proc/19557/stat : 19557 (concrete.sh) S 19555 19557 17668 0 -1 4194304 315 128 0 0 0 0 0 0 21 0 1 0 43988726 5484544 242 996147200 4194304 4889804 548682068752 18446744073709551615 272467813188 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/19557/statm: 1339 242 201 169 0 50 0
[pid=19560] ppid=19557 vsize=846840 CPUtime=1.48
/proc/19560/stat : 19560 (java) S 19557 19557 17668 0 -1 0 8290 0 1 0 141 7 0 0 18 0 10 0 43988727 867164160 7418 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 0 0 16800975 18446744073709551615 0 0 17 1 0 0
/proc/19560/statm: 211710 7418 2574 9 0 206270 0
[pid=19560/tid=19561] ppid=19557 vsize=846840 CPUtime=0.93
/proc/19560/task/19561/stat : 19561 (java) R 19557 19557 17668 0 -1 64 3462 0 1 0 89 4 0 0 24 0 10 0 43988728 867164160 7418 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19562] ppid=19557 vsize=846840 CPUtime=0.13
/proc/19560/task/19562/stat : 19562 (java) R 19557 19557 17668 0 -1 64 3040 0 0 0 12 1 0 0 16 0 10 0 43988729 867164160 7418 996147200 134512640 134550740 4294956000 18446744073709551615 102721892 0 0 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19563] ppid=19557 vsize=846840 CPUtime=0
/proc/19560/task/19563/stat : 19563 (java) S 19557 19557 17668 0 -1 64 13 0 0 0 0 0 0 0 16 0 10 0 43988729 867164160 7418 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19564] ppid=19557 vsize=846840 CPUtime=0
/proc/19560/task/19564/stat : 19564 (java) S 19557 19557 17668 0 -1 64 6 0 0 0 0 0 0 0 15 0 10 0 43988729 867164160 7418 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19565] ppid=19557 vsize=846840 CPUtime=0
/proc/19560/task/19565/stat : 19565 (java) S 19557 19557 17668 0 -1 64 1 0 0 0 0 0 0 0 18 0 10 0 43988731 867164160 7418 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19566] ppid=19557 vsize=846840 CPUtime=0.4
/proc/19560/task/19566/stat : 19566 (java) S 19557 19557 17668 0 -1 64 1050 0 0 0 39 1 0 0 15 0 10 0 43988731 867164160 7418 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19567] ppid=19557 vsize=846840 CPUtime=0
/proc/19560/task/19567/stat : 19567 (java) S 19557 19557 17668 0 -1 64 1 0 0 0 0 0 0 0 19 0 10 0 43988731 867164160 7418 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19568] ppid=19557 vsize=846840 CPUtime=0
/proc/19560/task/19568/stat : 19568 (java) S 19557 19557 17668 0 -1 64 3 0 0 0 0 0 0 0 15 0 10 0 43988731 867164160 7418 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19569] ppid=19557 vsize=846840 CPUtime=0
/proc/19560/task/19569/stat : 19569 (java) S 19557 19557 17668 0 -1 64 4 0 0 0 0 0 0 0 19 0 10 0 43988791 867164160 7418 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 852196

[startup+3.10171 s]
/proc/loadavg: 2.05 2.00 2.00 3/91 19569
/proc/meminfo: memFree=1788596/2055920 swapFree=4192812/4192956
[pid=19557] ppid=19555 vsize=5356 CPUtime=0
/proc/19557/stat : 19557 (concrete.sh) S 19555 19557 17668 0 -1 4194304 315 128 0 0 0 0 0 0 21 0 1 0 43988726 5484544 242 996147200 4194304 4889804 548682068752 18446744073709551615 272467813188 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/19557/statm: 1339 242 201 169 0 50 0
[pid=19560] ppid=19557 vsize=846996 CPUtime=3.06
/proc/19560/stat : 19560 (java) S 19557 19557 17668 0 -1 0 14826 0 1 0 295 11 0 0 18 0 9 0 43988727 867323904 13932 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 0 0 16800975 18446744073709551615 0 0 17 1 0 0
/proc/19560/statm: 211749 13932 2578 9 0 206309 0
[pid=19560/tid=19561] ppid=19557 vsize=846996 CPUtime=2.12
/proc/19560/task/19561/stat : 19561 (java) R 19557 19557 17668 0 -1 64 3814 0 1 0 208 4 0 0 25 0 9 0 43988728 867323904 13932 996147200 134512640 134550740 4294956000 18446744073709551615 4126857391 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19562] ppid=19557 vsize=846996 CPUtime=0.46
/proc/19560/task/19562/stat : 19562 (java) S 19557 19557 17668 0 -1 64 9160 0 0 0 42 4 0 0 17 0 9 0 43988729 867323904 13932 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 0 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19563] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19563/stat : 19563 (java) S 19557 19557 17668 0 -1 64 13 0 0 0 0 0 0 0 16 0 9 0 43988729 867323904 13932 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19564] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19564/stat : 19564 (java) S 19557 19557 17668 0 -1 64 6 0 0 0 0 0 0 0 15 0 9 0 43988729 867323904 13932 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19565] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19565/stat : 19565 (java) S 19557 19557 17668 0 -1 64 1 0 0 0 0 0 0 0 18 0 9 0 43988731 867323904 13932 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19566] ppid=19557 vsize=846996 CPUtime=0.44
/proc/19560/task/19566/stat : 19566 (java) S 19557 19557 17668 0 -1 64 1113 0 0 0 43 1 0 0 15 0 9 0 43988731 867323904 13932 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19567] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19567/stat : 19567 (java) S 19557 19557 17668 0 -1 64 1 0 0 0 0 0 0 0 19 0 9 0 43988731 867323904 13932 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19568] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19568/stat : 19568 (java) S 19557 19557 17668 0 -1 64 3 0 0 0 0 0 0 0 15 0 9 0 43988731 867323904 13932 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 3.06
Current children cumulated vsize (KiB) 852352

[startup+6.30293 s]
/proc/loadavg: 2.05 2.00 2.00 3/91 19569
/proc/meminfo: memFree=1788404/2055920 swapFree=4192812/4192956
[pid=19557] ppid=19555 vsize=5356 CPUtime=0
/proc/19557/stat : 19557 (concrete.sh) S 19555 19557 17668 0 -1 4194304 315 128 0 0 0 0 0 0 21 0 1 0 43988726 5484544 242 996147200 4194304 4889804 548682068752 18446744073709551615 272467813188 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/19557/statm: 1339 242 201 169 0 50 0
[pid=19560] ppid=19557 vsize=846996 CPUtime=6.23
/proc/19560/stat : 19560 (java) S 19557 19557 17668 0 -1 0 14834 0 1 0 612 11 0 0 18 0 9 0 43988727 867323904 13939 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 0 0 16800975 18446744073709551615 0 0 17 1 0 0
/proc/19560/statm: 211749 13939 2578 9 0 206309 0
[pid=19560/tid=19561] ppid=19557 vsize=846996 CPUtime=5.27
/proc/19560/task/19561/stat : 19561 (java) R 19557 19557 17668 0 -1 64 3814 0 1 0 523 4 0 0 25 0 9 0 43988728 867323904 13939 996147200 134512640 134550740 4294956000 18446744073709551615 4126846067 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19562] ppid=19557 vsize=846996 CPUtime=0.46
/proc/19560/task/19562/stat : 19562 (java) S 19557 19557 17668 0 -1 64 9160 0 0 0 42 4 0 0 16 0 9 0 43988729 867323904 13939 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 0 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19563] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19563/stat : 19563 (java) S 19557 19557 17668 0 -1 64 13 0 0 0 0 0 0 0 16 0 9 0 43988729 867323904 13939 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19564] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19564/stat : 19564 (java) S 19557 19557 17668 0 -1 64 6 0 0 0 0 0 0 0 15 0 9 0 43988729 867323904 13939 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19565] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19565/stat : 19565 (java) S 19557 19557 17668 0 -1 64 1 0 0 0 0 0 0 0 18 0 9 0 43988731 867323904 13939 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19566] ppid=19557 vsize=846996 CPUtime=0.46
/proc/19560/task/19566/stat : 19566 (java) S 19557 19557 17668 0 -1 64 1121 0 0 0 45 1 0 0 15 0 9 0 43988731 867323904 13939 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19567] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19567/stat : 19567 (java) S 19557 19557 17668 0 -1 64 1 0 0 0 0 0 0 0 19 0 9 0 43988731 867323904 13939 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19568] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19568/stat : 19568 (java) S 19557 19557 17668 0 -1 64 3 0 0 0 0 0 0 0 15 0 9 0 43988731 867323904 13939 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 6.23
Current children cumulated vsize (KiB) 852352

[startup+12.7014 s]
/proc/loadavg: 2.04 2.00 2.00 3/91 19569
/proc/meminfo: memFree=1788404/2055920 swapFree=4192812/4192956
[pid=19557] ppid=19555 vsize=5356 CPUtime=0
/proc/19557/stat : 19557 (concrete.sh) S 19555 19557 17668 0 -1 4194304 315 128 0 0 0 0 0 0 21 0 1 0 43988726 5484544 242 996147200 4194304 4889804 548682068752 18446744073709551615 272467813188 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/19557/statm: 1339 242 201 169 0 50 0
[pid=19560] ppid=19557 vsize=846996 CPUtime=12.56
/proc/19560/stat : 19560 (java) S 19557 19557 17668 0 -1 0 14834 0 1 0 1245 11 0 0 18 0 9 0 43988727 867323904 13939 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 0 0 16800975 18446744073709551615 0 0 17 1 0 0
/proc/19560/statm: 211749 13939 2578 9 0 206309 0
[pid=19560/tid=19561] ppid=19557 vsize=846996 CPUtime=11.61
/proc/19560/task/19561/stat : 19561 (java) R 19557 19557 17668 0 -1 64 3814 0 1 0 1157 4 0 0 25 0 9 0 43988728 867323904 13939 996147200 134512640 134550740 4294956000 18446744073709551615 4126838862 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19562] ppid=19557 vsize=846996 CPUtime=0.46
/proc/19560/task/19562/stat : 19562 (java) S 19557 19557 17668 0 -1 64 9160 0 0 0 42 4 0 0 16 0 9 0 43988729 867323904 13939 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 0 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19563] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19563/stat : 19563 (java) S 19557 19557 17668 0 -1 64 13 0 0 0 0 0 0 0 16 0 9 0 43988729 867323904 13939 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19564] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19564/stat : 19564 (java) S 19557 19557 17668 0 -1 64 6 0 0 0 0 0 0 0 15 0 9 0 43988729 867323904 13939 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19565] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19565/stat : 19565 (java) S 19557 19557 17668 0 -1 64 1 0 0 0 0 0 0 0 18 0 9 0 43988731 867323904 13939 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19566] ppid=19557 vsize=846996 CPUtime=0.46
/proc/19560/task/19566/stat : 19566 (java) S 19557 19557 17668 0 -1 64 1121 0 0 0 45 1 0 0 15 0 9 0 43988731 867323904 13939 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19567] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19567/stat : 19567 (java) S 19557 19557 17668 0 -1 64 1 0 0 0 0 0 0 0 19 0 9 0 43988731 867323904 13939 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19568] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19568/stat : 19568 (java) S 19557 19557 17668 0 -1 64 3 0 0 0 0 0 0 0 15 0 9 0 43988731 867323904 13939 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 12.56
Current children cumulated vsize (KiB) 852352

[startup+25.5022 s]
/proc/loadavg: 2.03 2.00 2.00 3/91 19569
/proc/meminfo: memFree=1788340/2055920 swapFree=4192812/4192956
[pid=19557] ppid=19555 vsize=5356 CPUtime=0
/proc/19557/stat : 19557 (concrete.sh) S 19555 19557 17668 0 -1 4194304 315 128 0 0 0 0 0 0 21 0 1 0 43988726 5484544 242 996147200 4194304 4889804 548682068752 18446744073709551615 272467813188 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/19557/statm: 1339 242 201 169 0 50 0
[pid=19560] ppid=19557 vsize=846996 CPUtime=25.24
/proc/19560/stat : 19560 (java) S 19557 19557 17668 0 -1 0 14835 0 1 0 2513 11 0 0 18 0 9 0 43988727 867323904 13940 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 0 0 16800975 18446744073709551615 0 0 17 1 0 0
/proc/19560/statm: 211749 13940 2578 9 0 206309 0
[pid=19560/tid=19561] ppid=19557 vsize=846996 CPUtime=24.29
/proc/19560/task/19561/stat : 19561 (java) R 19557 19557 17668 0 -1 64 3814 0 1 0 2424 5 0 0 25 0 9 0 43988728 867323904 13940 996147200 134512640 134550740 4294956000 18446744073709551615 4126855293 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19562] ppid=19557 vsize=846996 CPUtime=0.46
/proc/19560/task/19562/stat : 19562 (java) S 19557 19557 17668 0 -1 64 9160 0 0 0 42 4 0 0 16 0 9 0 43988729 867323904 13940 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 0 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19563] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19563/stat : 19563 (java) S 19557 19557 17668 0 -1 64 13 0 0 0 0 0 0 0 16 0 9 0 43988729 867323904 13940 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19564] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19564/stat : 19564 (java) S 19557 19557 17668 0 -1 64 6 0 0 0 0 0 0 0 15 0 9 0 43988729 867323904 13940 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19565] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19565/stat : 19565 (java) S 19557 19557 17668 0 -1 64 1 0 0 0 0 0 0 0 18 0 9 0 43988731 867323904 13940 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19566] ppid=19557 vsize=846996 CPUtime=0.46
/proc/19560/task/19566/stat : 19566 (java) S 19557 19557 17668 0 -1 64 1122 0 0 0 45 1 0 0 15 0 9 0 43988731 867323904 13940 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19567] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19567/stat : 19567 (java) S 19557 19557 17668 0 -1 64 1 0 0 0 0 0 0 0 19 0 9 0 43988731 867323904 13940 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19568] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19568/stat : 19568 (java) S 19557 19557 17668 0 -1 64 3 0 0 0 0 0 0 0 15 0 9 0 43988731 867323904 13940 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 25.24
Current children cumulated vsize (KiB) 852352

################
# More data... #
################

/proc/19560/task/19566/stat : 19566 (java) S 19557 19557 17668 0 -1 64 1131 0 0 0 47 1 0 0 16 0 9 0 43988731 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19567] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19567/stat : 19567 (java) S 19557 19557 17668 0 -1 64 1 0 0 0 0 0 0 0 19 0 9 0 43988731 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19568] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19568/stat : 19568 (java) S 19557 19557 17668 0 -1 64 3 0 0 0 0 0 0 0 15 0 9 0 43988731 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 3317.91
Current children cumulated vsize (KiB) 852352

[startup+3402.31 s]
/proc/loadavg: 2.35 2.19 2.08 3/92 19676
/proc/meminfo: memFree=650548/2055920 swapFree=4192812/4192956
[pid=19557] ppid=19555 vsize=5356 CPUtime=0
/proc/19557/stat : 19557 (concrete.sh) S 19555 19557 17668 0 -1 4194304 315 128 0 0 0 0 0 0 21 0 1 0 43988726 5484544 242 996147200 4194304 4889804 548682068752 18446744073709551615 272467813188 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/19557/statm: 1339 242 201 169 0 50 0
[pid=19560] ppid=19557 vsize=846996 CPUtime=3377.88
/proc/19560/stat : 19560 (java) S 19557 19557 17668 0 -1 0 14994 0 1 0 337703 85 0 0 18 0 9 0 43988727 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 0 0 16800975 18446744073709551615 0 0 17 1 0 0
/proc/19560/statm: 211749 14090 2578 9 0 206309 0
[pid=19560/tid=19561] ppid=19557 vsize=846996 CPUtime=3376.91
/proc/19560/task/19561/stat : 19561 (java) R 19557 19557 17668 0 -1 64 3951 0 1 0 337612 79 0 0 25 0 9 0 43988728 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4126846159 0 4 0 16800975 0 0 0 -1 0 0 0
[pid=19560/tid=19562] ppid=19557 vsize=846996 CPUtime=0.46
/proc/19560/task/19562/stat : 19562 (java) S 19557 19557 17668 0 -1 64 9173 0 0 0 42 4 0 0 16 0 9 0 43988729 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 0 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19563] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19563/stat : 19563 (java) S 19557 19557 17668 0 -1 64 13 0 0 0 0 0 0 0 16 0 9 0 43988729 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19564] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19564/stat : 19564 (java) S 19557 19557 17668 0 -1 64 6 0 0 0 0 0 0 0 15 0 9 0 43988729 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19565] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19565/stat : 19565 (java) S 19557 19557 17668 0 -1 64 1 0 0 0 0 0 0 0 18 0 9 0 43988731 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19566] ppid=19557 vsize=846996 CPUtime=0.48
/proc/19560/task/19566/stat : 19566 (java) S 19557 19557 17668 0 -1 64 1131 0 0 0 47 1 0 0 16 0 9 0 43988731 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19567] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19567/stat : 19567 (java) S 19557 19557 17668 0 -1 64 1 0 0 0 0 0 0 0 19 0 9 0 43988731 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19568] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19568/stat : 19568 (java) S 19557 19557 17668 0 -1 64 3 0 0 0 0 0 0 0 15 0 9 0 43988731 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 3377.88
Current children cumulated vsize (KiB) 852352

[startup+3462.3 s]
/proc/loadavg: 2.61 2.28 2.12 3/92 19676
/proc/meminfo: memFree=361524/2055920 swapFree=4192812/4192956
[pid=19557] ppid=19555 vsize=5356 CPUtime=0
/proc/19557/stat : 19557 (concrete.sh) S 19555 19557 17668 0 -1 4194304 315 128 0 0 0 0 0 0 21 0 1 0 43988726 5484544 242 996147200 4194304 4889804 548682068752 18446744073709551615 272467813188 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/19557/statm: 1339 242 201 169 0 50 0
[pid=19560] ppid=19557 vsize=846996 CPUtime=3437.84
/proc/19560/stat : 19560 (java) S 19557 19557 17668 0 -1 0 14994 0 1 0 343693 91 0 0 18 0 9 0 43988727 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 0 0 16800975 18446744073709551615 0 0 17 1 0 0
/proc/19560/statm: 211749 14090 2578 9 0 206309 0
[pid=19560/tid=19561] ppid=19557 vsize=846996 CPUtime=3436.87
/proc/19560/task/19561/stat : 19561 (java) R 19557 19557 17668 0 -1 64 3951 0 1 0 343603 84 0 0 25 0 9 0 43988728 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4126838809 0 4 0 16800975 0 0 0 -1 0 0 0
[pid=19560/tid=19562] ppid=19557 vsize=846996 CPUtime=0.46
/proc/19560/task/19562/stat : 19562 (java) S 19557 19557 17668 0 -1 64 9173 0 0 0 42 4 0 0 16 0 9 0 43988729 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 0 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19563] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19563/stat : 19563 (java) S 19557 19557 17668 0 -1 64 13 0 0 0 0 0 0 0 16 0 9 0 43988729 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19564] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19564/stat : 19564 (java) S 19557 19557 17668 0 -1 64 6 0 0 0 0 0 0 0 15 0 9 0 43988729 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19565] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19565/stat : 19565 (java) S 19557 19557 17668 0 -1 64 1 0 0 0 0 0 0 0 18 0 9 0 43988731 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19566] ppid=19557 vsize=846996 CPUtime=0.48
/proc/19560/task/19566/stat : 19566 (java) S 19557 19557 17668 0 -1 64 1131 0 0 0 47 1 0 0 16 0 9 0 43988731 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19567] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19567/stat : 19567 (java) S 19557 19557 17668 0 -1 64 1 0 0 0 0 0 0 0 19 0 9 0 43988731 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19568] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19568/stat : 19568 (java) S 19557 19557 17668 0 -1 64 3 0 0 0 0 0 0 0 15 0 9 0 43988731 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 3437.84
Current children cumulated vsize (KiB) 852352

[startup+3522.3 s]
/proc/loadavg: 2.89 2.42 2.17 3/92 19676
/proc/meminfo: memFree=63092/2055920 swapFree=4192812/4192956
[pid=19557] ppid=19555 vsize=5356 CPUtime=0
/proc/19557/stat : 19557 (concrete.sh) S 19555 19557 17668 0 -1 4194304 315 128 0 0 0 0 0 0 21 0 1 0 43988726 5484544 242 996147200 4194304 4889804 548682068752 18446744073709551615 272467813188 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/19557/statm: 1339 242 201 169 0 50 0
[pid=19560] ppid=19557 vsize=846996 CPUtime=3497.8
/proc/19560/stat : 19560 (java) S 19557 19557 17668 0 -1 0 14994 0 1 0 349682 98 0 0 18 0 9 0 43988727 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 0 0 16800975 18446744073709551615 0 0 17 1 0 0
/proc/19560/statm: 211749 14090 2578 9 0 206309 0
[pid=19560/tid=19561] ppid=19557 vsize=846996 CPUtime=3496.82
/proc/19560/task/19561/stat : 19561 (java) R 19557 19557 17668 0 -1 64 3951 0 1 0 349591 91 0 0 25 0 9 0 43988728 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4126855213 0 4 0 16800975 0 0 0 -1 0 0 0
[pid=19560/tid=19562] ppid=19557 vsize=846996 CPUtime=0.46
/proc/19560/task/19562/stat : 19562 (java) S 19557 19557 17668 0 -1 64 9173 0 0 0 42 4 0 0 16 0 9 0 43988729 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 0 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19563] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19563/stat : 19563 (java) S 19557 19557 17668 0 -1 64 13 0 0 0 0 0 0 0 16 0 9 0 43988729 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19564] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19564/stat : 19564 (java) S 19557 19557 17668 0 -1 64 6 0 0 0 0 0 0 0 15 0 9 0 43988729 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19565] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19565/stat : 19565 (java) S 19557 19557 17668 0 -1 64 1 0 0 0 0 0 0 0 18 0 9 0 43988731 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19566] ppid=19557 vsize=846996 CPUtime=0.48
/proc/19560/task/19566/stat : 19566 (java) S 19557 19557 17668 0 -1 64 1131 0 0 0 47 1 0 0 16 0 9 0 43988731 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19567] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19567/stat : 19567 (java) S 19557 19557 17668 0 -1 64 1 0 0 0 0 0 0 0 19 0 9 0 43988731 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19568] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19568/stat : 19568 (java) S 19557 19557 17668 0 -1 64 3 0 0 0 0 0 0 0 15 0 9 0 43988731 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 3497.8
Current children cumulated vsize (KiB) 852352

[startup+3582.3 s]
/proc/loadavg: 2.96 2.52 2.22 3/92 19676
/proc/meminfo: memFree=14900/2055920 swapFree=4192812/4192956
[pid=19557] ppid=19555 vsize=5356 CPUtime=0
/proc/19557/stat : 19557 (concrete.sh) S 19555 19557 17668 0 -1 4194304 315 128 0 0 0 0 0 0 21 0 1 0 43988726 5484544 242 996147200 4194304 4889804 548682068752 18446744073709551615 272467813188 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/19557/statm: 1339 242 201 169 0 50 0
[pid=19560] ppid=19557 vsize=846996 CPUtime=3557.68
/proc/19560/stat : 19560 (java) S 19557 19557 17668 0 -1 0 14994 0 1 0 355665 103 0 0 18 0 9 0 43988727 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 0 0 16800975 18446744073709551615 0 0 17 1 0 0
/proc/19560/statm: 211749 14090 2578 9 0 206309 0
[pid=19560/tid=19561] ppid=19557 vsize=846996 CPUtime=3556.71
/proc/19560/task/19561/stat : 19561 (java) R 19557 19557 17668 0 -1 64 3951 0 1 0 355575 96 0 0 25 0 9 0 43988728 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4126862555 0 4 0 16800975 0 0 0 -1 0 0 0
[pid=19560/tid=19562] ppid=19557 vsize=846996 CPUtime=0.46
/proc/19560/task/19562/stat : 19562 (java) S 19557 19557 17668 0 -1 64 9173 0 0 0 42 4 0 0 16 0 9 0 43988729 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 0 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19563] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19563/stat : 19563 (java) S 19557 19557 17668 0 -1 64 13 0 0 0 0 0 0 0 16 0 9 0 43988729 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19564] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19564/stat : 19564 (java) S 19557 19557 17668 0 -1 64 6 0 0 0 0 0 0 0 15 0 9 0 43988729 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19565] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19565/stat : 19565 (java) S 19557 19557 17668 0 -1 64 1 0 0 0 0 0 0 0 18 0 9 0 43988731 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19566] ppid=19557 vsize=846996 CPUtime=0.48
/proc/19560/task/19566/stat : 19566 (java) S 19557 19557 17668 0 -1 64 1131 0 0 0 47 1 0 0 16 0 9 0 43988731 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19567] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19567/stat : 19567 (java) S 19557 19557 17668 0 -1 64 1 0 0 0 0 0 0 0 19 0 9 0 43988731 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19568] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19568/stat : 19568 (java) S 19557 19557 17668 0 -1 64 3 0 0 0 0 0 0 0 15 0 9 0 43988731 867323904 14090 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 3557.68
Current children cumulated vsize (KiB) 852352



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+3624.9 s]
/proc/loadavg: 2.98 2.58 2.25 4/90 19711
/proc/meminfo: memFree=1789828/2055920 swapFree=4192812/4192956
[pid=19557] ppid=19555 vsize=5356 CPUtime=0
/proc/19557/stat : 19557 (concrete.sh) S 19555 19557 17668 0 -1 4194304 315 128 0 0 0 0 0 0 21 0 1 0 43988726 5484544 242 996147200 4194304 4889804 548682068752 18446744073709551615 272467813188 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/19557/statm: 1339 242 201 169 0 50 0
[pid=19560] ppid=19557 vsize=846996 CPUtime=3600.01
/proc/19560/stat : 19560 (java) S 19557 19557 17668 0 -1 0 14995 0 1 0 359888 113 0 0 18 0 9 0 43988727 867323904 14091 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 0 0 16800975 18446744073709551615 0 0 17 1 0 0
/proc/19560/statm: 211749 14091 2578 9 0 206309 0
[pid=19560/tid=19561] ppid=19557 vsize=846996 CPUtime=3599.05
/proc/19560/task/19561/stat : 19561 (java) R 19557 19557 17668 0 -1 64 3951 0 1 0 359798 107 0 0 25 0 9 0 43988728 867323904 14091 996147200 134512640 134550740 4294956000 18446744073709551615 4126862571 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19562] ppid=19557 vsize=846996 CPUtime=0.46
/proc/19560/task/19562/stat : 19562 (java) S 19557 19557 17668 0 -1 64 9173 0 0 0 42 4 0 0 16 0 9 0 43988729 867323904 14091 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 0 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19563] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19563/stat : 19563 (java) S 19557 19557 17668 0 -1 64 13 0 0 0 0 0 0 0 16 0 9 0 43988729 867323904 14091 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19564] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19564/stat : 19564 (java) S 19557 19557 17668 0 -1 64 6 0 0 0 0 0 0 0 15 0 9 0 43988729 867323904 14091 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19565] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19565/stat : 19565 (java) S 19557 19557 17668 0 -1 64 1 0 0 0 0 0 0 0 18 0 9 0 43988731 867323904 14091 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19566] ppid=19557 vsize=846996 CPUtime=0.48
/proc/19560/task/19566/stat : 19566 (java) S 19557 19557 17668 0 -1 64 1132 0 0 0 47 1 0 0 16 0 9 0 43988731 867323904 14091 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 0 0 0
[pid=19560/tid=19567] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19567/stat : 19567 (java) S 19557 19557 17668 0 -1 64 1 0 0 0 0 0 0 0 19 0 9 0 43988731 867323904 14091 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 1 0 0
[pid=19560/tid=19568] ppid=19557 vsize=846996 CPUtime=0
/proc/19560/task/19568/stat : 19568 (java) S 19557 19557 17668 0 -1 64 3 0 0 0 0 0 0 0 15 0 9 0 43988731 867323904 14091 996147200 134512640 134550740 4294956000 18446744073709551615 4294960144 0 4 0 16800975 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 3600.01
Current children cumulated vsize (KiB) 852352

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 19557 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=1999
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=1999
# CPU time returned by wait4() is 0.003998
# while last known CPU time is 3600.01
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 3624.92
CPU time (s): 3600.01
CPU user time (s): 3598.88
CPU system time (s): 1.13
CPU usage (%): 99.3128
Max. virtual memory (cumulated for all children) (KiB): 852352

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.001999
system time used= 0.001999
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 443
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 8
involuntary context switches= 5

runsolver used 5.34419 second user time and 11.2393 second system time

The end

Launcher Data (download as text)

Begin job on node22 at 2008-07-12 16:27:42
IDJOB=1112705
IDBENCH=61461
IDSOLVER=335
FILE ID=node22/1112705-1215872862
PBS_JOBID= 7903442
Free space on /tmp= 66464 MiB

SOLVER NAME= Concrete + CSP4J - MCRW Engine 2008-05-30
BENCH NAME= CPAI08/csp/os-gp/normalized-gp10-05-1070.xml
COMMAND LINE= HOME/concrete.sh -c -ext 0 -s MCRW -max BENCHNAME 
RUNSOLVER COMMAND LINE= /home/evaluation/evaluation/pub/bin/runsolver --timestamp -w /tmp/evaluation-result-1112705-1215872862/watcher-1112705-1215872862 -o /tmp/evaluation-result-1112705-1215872862/solver-1112705-1215872862 -C 3600 -W 4000 -M 900 --output-limit 1,15  HOME/concrete.sh -c -ext 0 -s MCRW -max HOME/instance-1112705-1215872862.xml 

TIME LIMIT= 3600 seconds
MEMORY LIMIT= 900 MiB

MD5SUM BENCH= 76853a39367362e3e4e459176be1a8ba
RANDOM SEED=66338555

node22.alineos.net Linux 2.6.9-22.EL.rootsmp #1 SMP Mon Oct 3 08:59:52 CEST 2005

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.253
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5914.62
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.253
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5586.94
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:      2055920 kB
MemFree:       1835180 kB
Buffers:         52864 kB
Cached:          91364 kB
SwapCached:          0 kB
Active:          97812 kB
Inactive:        76688 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      2055920 kB
LowFree:       1835180 kB
SwapTotal:     4192956 kB
SwapFree:      4192812 kB
Dirty:            1084 kB
Writeback:           0 kB
Mapped:          50416 kB
Slab:            31380 kB
Committed_AS:   230772 kB
PageTables:       1880 kB
VmallocTotal: 536870911 kB
VmallocUsed:    264952 kB
VmallocChunk: 536605679 kB
HugePages_Total:     0
HugePages_Free:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 66452 MiB
End job on node22 at 2008-07-12 17:28:09