Trace number 2083134

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Concrete 2009-07-14SAT 112.744 113.881

DiagnosticValue
CHECKS9038020
NODES1765

General information on the benchmark

Namecsp/fapp/fapp10/
normalized-fapp10-0900-9.xml
MD5SUMbb8fed9f370e164b93c37276047ba0d0
Bench Category2-ARY-INT (binary constraints in intension)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark64.8031
Satisfiable
(Un)Satisfiability was proved
Number of variables900
Number of constraints11381
Maximum constraint arity2
Maximum domain size362
Number of constraints which are defined in extension0
Number of constraints which are defined in intension11381
Global constraints used (with number of constraints)

Solver Data 

90.09/91.00	c CPU-LOAD 75.24
90.09/91.00	c LOADED 900 V (362 E), 6071 ExtensionConstraint2D, arity 2, 75.24 cpu
112.58/113.77	v -2604 -2608 -2252 -2368 2564 -2308 -2340 -2700 2448 2512 -2284 -2456 -2544 -2420 -2504 -2372 2556 2188 2040 -2576 2584 2624 -2728 2628 2660 -2060 -2672 2316 -2608 2524 -2408 2264 -2512 -2268 -2700 -2472 2604 -2528 -2580 2188 2084 -2384 2544 2432 -2520 -2104 2228 -2532 2308 -2656 -2220 -2532 2300 -2220 -2064 2380 -2436 -2708 -2740 -2308 -2616 -2688 -2104 -2632 -2332 -2528 -2704 2064 -2380 -2688 -2648 2168 -2316 -2244 -2656 -2712 -2332 -2572 2048 -2632 2628 2452 2688 2144 2256 -2208 -2508 -2256 2716 -2484 -2516 -2692 -2476 -2720 2084 -2488 2448 2540 -2584 2468 2688 2532 -2348 2512 -2344 -2688 2720 2684 2548 -2700 2336 -2560 -2720 -2656 -2400 -2184 -2020 -2568 -2628 -2040 -2224 2324 2480 2144 -2296 -2696 -2692 2060 -2528 -2068 2196 -2544 -2248 2408 -2344 -2524 2104 -2408 -2672 -2524 -2524 -2268 -2284 2344 -2680 -2536 2556 -2448 -2728 -2632 -2464 -2360 -2484 2092 -2040 2672 2700 -2308 -2144 2116 2252 -2540 -2680 -2516 2672 2136 -2060 2036 -2296 -2092 -2584 -2516 -2068 -2052 -2304 2516 -2740 -2180 -2604 2296 2488 -2344 -2688 -2624 -2572 2104 2020 2220 2532 2460 2476 -2416 -2324 -2224 2652 -2572 2552 2344 2312 -2692 2164 -2428 -2420 2712 2624 2548 -2436 -2144 2124 -2540 2292 -2168 2324 -2460 -2716 -2436 2056 2580 -2524 2412 -2384 -2256 -2464 2620 2276 -2196 -2216 -2324 2348 2228 2488 2480 -2060 -2700 -2344 -2280 -2440 2384 2324 -2620 -2364 -2640 -2724 -2256 2584 -2616 -2644 2408 2416 -2644 -2280 2496 -2204 -2680 -2324 -2544 -2320 -2532 -2716 -2604 -2288 -2452 -2332 2460 -2580 -2076 -2544 -2444 -2484 2212 2508 -2536 -2260 -2540 -2620 2412 -2492 -2456 2168 -2376 2692 -2280 -2484 2520 2228 -2648 -2688 2324 -2224 2324 -2692 2692 -2656 2068 -2312 2612 2572 2428 -2268 -2080 2340 2236 -2600 2428 2492 2336 -2340 2468 -2028 2120 -2420 2596 2644 -2732 -2468 2136 2584 -2496 -2680 2176 2700 2660 -2464 2468 -2700 -2492 2596 2672 -2604 2648 -2216 -2652 2680 -2120 -2380 -2020 -2724 2560 -2544 -2420 -2256 -2468 2616 -2500 -2520 -2592 -2456 -2060 2408 2492 2612 -2480 -2060 -2528 2392 -2388 -2340 -2196 -2196 -2504 -2364 -2360 2128 -2448 2020 -2604 -2316 -2060 -2180 -2304 -2060 -2720 2672 2500 -2040 -2552 -2432 -2652 2592 2020 2336 -2692 2392 -2652 -2596 -2024 -2060 2228 2668 -2720 2472 -2188 2528 2020 2560 -2184 2020 -2668 2572 2020 2104 2504 -2488 2272 -2596 -2740 2028 -2712 2068 -2580 -2492 -2484 -2388 -2636 2672 2348 -2432 2304 2580 -2708 -2348 -2156 -2620 2604 2060 -2660 -2544 -2312 -2048 2604 -2592 -2176 -2116 -2304 2644 2080 -2592 -2336 -2700 -2640 2500 2556 2096 -2496 -2712 -2428 2120 -2656 -2656 -2524 -2408 -2516 -2416 -2456 2572 -2064 -2104 2568 2488 -2264 -2740 2316 -2548 2368 -2672 2068 2140 2680 -2680 -2712 -2284 -2156 -2532 2080 2448 2376 -2580 2376 2236 2420 2440 2064 -2308 2060 2284 -2740 -2656 -2504 -2516 2508 2636 2452 -2600 -2380 -2352 2692 -2524 -2276 2536 -2644 2276 -2740 -2380 -2732 -2284 -2620 2056 -2208 -2312 -2696 2020 2512 -2248 2408 -2632 -2592 -2652 2668 2216 2720 -2516 -2436 -2072 2272 2036 2364 -2564 2476 -2508 -2468 -2184 -2508 -2452 -2384 -2700 -2492 2216 2564 -2384 -2380 2688 2184 -2020 2104 -2720 2292 -2484 -2612 -2740 2648 -2204 -2332 -2264 -2060 -2096 2448 2540 2680 -2080 2148 -2644 2728 -2580 2284 -2632 2668 -2700 -2504 -2556 -2672 2536 -2484 -2296 -2680 -2324 2184 2644 -2308 -2480 -2508 -2252 -2356 -2316 -2656 2620 2020 -2156 -2640 2312 -2572 -2680 -2364 2540 -2596 -2644 2020 2676 -2064 -2420 -2060 2560 -2284 -2544 -2620 -2660 -2080 2144 2444 -2564 2548 2260 -2364 2432 -2024 -2672 -2544 -2364 2616 -2584 -2108 -2656 -2228 -2424 -2716 2212 2028 -2492 -2224 -2644 -2124 2640 -2208 2652 2564 -2108 2188 2020 -2728 -2548 2636 2388 -2552 -2504 -2188 -2564 2680 2484 -2516 -2380 2492 2572 2024 -2656 2340 -2604 -2588 2444 -2080 -2556 -2276 2188 -2712 -2388 -2644 2100 2532 -2588 2600 -2464 -2664 -2624 2696 -2028 2604 2444 -2372 -2692 -2388 -2664 -2708 -2632 -2344 -2624 -2380 -2020 -2740 2144 2392 2712 -2300 -2108 -2668 -2420 2660 2140 -2588 2676 -2396 -2508 2564 2632 -2700 2572 -2236 2708 -2508 2184 -2564 -2080 2052 -2136 2508 -2304 -2632 -2580 2556 2352 -2256 -2304 -2680 -2284 -2088 2532 -2096 -2552 -2156 -2612 -2532 -2564 -2548 -2196 2648 2524 -2356 -2372 -2444 2216 2176 -2288 2488 2528 -2672 2496 -2520 2636 -2740 -2476 -2080 -2324 -2648 2424 -2740 -2548 -2156 2268 -2320 2496 -2064 -2256 2556 2640 2060 -2712 -2620 -2072 -2092 -2584 2212 -2232 -2396 -2172 -2700 -2588 2460 -2476 -2732 2328 2532 -2184 -2272 -2528 2240 -2068 2516 2476 2688 2548 -2696 -2236 -2096 2324 -2476 -2684 2196 -2644 2660 -2344 -2672 2580 2632 2064 2364 2492 2320 -2548 2348 2216 -2088 -2408 -2324 -2684 -2700 -2740 -2616 -2344 2596 2616 -2540 -2540 -2720 -2564 2020 -2356 -2032 -2500 -2476 2256 -2384 -2676 -2220 2424 -2112 2488 2136 -2692 -2068 -2584 -2132 -2468 -2164 -2312 -2056 -2368 -2716 2676 -2648 2452 -2680 2432 -2572 2488 -2060 -2736 -2524 -2324 -2688 -2668 -2584 2604 -2472 -2664 -2684 -2644 -2608 -2640 -2696 -2704 -2528 -2396 -2676 2508 2524 2516 -2076 2372 -2556 -2684 2180 -2616 2720 -2244 2452 -2512 -2576 -2352 -2144 -2464 -2352 -2596 -2096 -2664 2604 -2040 -2732 2216 -2600 -2420 -2676 -2508 -2648 
112.58/113.77	s SATISFIABLE
112.58/113.77	d NODES 1765
112.58/113.77	d CHECKS 9038015
112.58/113.77	c heuristic-cpu: 0.0
112.58/113.77	c prepro-constraint-ccks: 0
112.58/113.77	c prepro-constraint-presenceccks: 0
112.58/113.77	c prepro-cpu: 12.68
112.58/113.77	c prepro-matrix2d-ccks: 747393
112.58/113.77	c prepro-matrix2d-presenceccks: 378406
112.58/113.77	c prepro-removed: 44560
112.58/113.77	c search-cpu: 8.34
112.58/113.77	c search-nps: 211.63069
112.58/113.77	c total-assgn: 1765
112.58/113.77	c total-constraint-ccks: 0
112.58/113.77	c total-constraint-presence-ccks: 0
112.58/113.77	c total-cpu: 21.51
112.58/113.77	c total-matrix2d-ccks: 9038015
112.58/113.77	c total-matrix2d-presence-ccks: 12365570
112.58/113.77	c TOTAL-NODES 1765
112.58/113.77	c TOTAL-CPU 21.51

Verifier Data 

OK

Watcher Data 

runsolver version 3.2.9 (svn:492) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-2083134-1247722516/watcher-2083134-1247722516 -o /tmp/evaluation-result-2083134-1247722516/solver-2083134-1247722516 -C 1800 -W 2000 -M 900 HOME/concrete.sh -c HOME/instance-2083134-1247722516.xml 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 921600 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 972800 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.01 2.03 2.00 4/72 12123
/proc/meminfo: memFree=1106100/2055920 swapFree=4192812/4192956
[pid=12123] ppid=12121 vsize=5356 CPUtime=0
/proc/12123/stat : 12123 (concrete.sh) S 12121 12123 8601 0 -1 4194304 299 127 0 0 0 0 0 0 20 0 1 0 59318748 5484544 238 996147200 4194304 4889804 548682068816 18446744073709551615 215419158610 0 0 4100 65536 18446744071563648864 0 0 17 1 0 0
/proc/12123/statm: 1339 238 197 169 0 50 0
[pid=12125] ppid=12123 vsize=5356 CPUtime=0
/proc/12125/stat : 12125 (concrete.sh) R 12123 12123 8601 0 -1 4194368 18 0 0 0 0 0 0 0 22 0 1 0 59318748 5484544 238 996147200 4194304 4889804 548682068816 18446744073709551615 4338417 0 0 4096 65538 0 0 0 17 1 0 0
/proc/12125/statm: 1339 238 197 169 0 50 0

[startup+0.037545 s]
/proc/loadavg: 2.01 2.03 2.00 4/72 12123
/proc/meminfo: memFree=1106100/2055920 swapFree=4192812/4192956
[pid=12123] ppid=12121 vsize=5356 CPUtime=0
/proc/12123/stat : 12123 (concrete.sh) S 12121 12123 8601 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59318748 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/12123/statm: 1339 244 201 169 0 50 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 5356

[startup+0.101429 s]
/proc/loadavg: 2.01 2.03 2.00 4/72 12123
/proc/meminfo: memFree=1106100/2055920 swapFree=4192812/4192956
[pid=12123] ppid=12121 vsize=5356 CPUtime=0
/proc/12123/stat : 12123 (concrete.sh) S 12121 12123 8601 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59318748 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/12123/statm: 1339 244 201 169 0 50 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 5356

[startup+0.301454 s]
/proc/loadavg: 2.01 2.03 2.00 4/72 12123
/proc/meminfo: memFree=1106100/2055920 swapFree=4192812/4192956
[pid=12123] ppid=12121 vsize=5356 CPUtime=0
/proc/12123/stat : 12123 (concrete.sh) S 12121 12123 8601 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59318748 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/12123/statm: 1339 244 201 169 0 50 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 5356

[startup+0.701498 s]
/proc/loadavg: 2.01 2.03 2.00 4/72 12123
/proc/meminfo: memFree=1106100/2055920 swapFree=4192812/4192956
[pid=12123] ppid=12121 vsize=5356 CPUtime=0
/proc/12123/stat : 12123 (concrete.sh) S 12121 12123 8601 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59318748 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/12123/statm: 1339 244 201 169 0 50 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 5356

[startup+1.50159 s]
/proc/loadavg: 2.01 2.03 2.00 3/82 12136
/proc/meminfo: memFree=1090596/2055920 swapFree=4192812/4192956
[pid=12123] ppid=12121 vsize=5356 CPUtime=0
/proc/12123/stat : 12123 (concrete.sh) S 12121 12123 8601 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59318748 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/12123/statm: 1339 244 201 169 0 50 0
[pid=12128] ppid=12123 vsize=848608 CPUtime=1.47
/proc/12128/stat : 12128 (java) S 12123 12123 8601 0 -1 0 8280 0 1 0 141 6 0 0 18 0 9 0 59318749 868974592 7116 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 18446744073709551615 0 0 17 1 0 0
/proc/12128/statm: 212152 7116 2624 9 0 206630 0
[pid=12128/tid=12129] ppid=12123 vsize=848608 CPUtime=0.98
/proc/12128/task/12129/stat : 12129 (java) R 12123 12123 8601 0 -1 64 3920 0 1 0 95 3 0 0 21 0 9 0 59318750 868974592 7116 996147200 134512640 134550932 4294955408 18446744073709551615 4125182144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12130] ppid=12123 vsize=848608 CPUtime=0.13
/proc/12128/task/12130/stat : 12130 (java) S 12123 12123 8601 0 -1 64 2120 0 0 0 13 0 0 0 16 0 9 0 59318751 868974592 7116 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12131] ppid=12123 vsize=848608 CPUtime=0
/proc/12128/task/12131/stat : 12131 (java) S 12123 12123 8601 0 -1 64 13 0 0 0 0 0 0 0 16 0 9 0 59318752 868974592 7116 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12132] ppid=12123 vsize=848608 CPUtime=0
/proc/12128/task/12132/stat : 12132 (java) S 12123 12123 8601 0 -1 64 4 0 0 0 0 0 0 0 15 0 9 0 59318752 868974592 7116 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12133] ppid=12123 vsize=848608 CPUtime=0
/proc/12128/task/12133/stat : 12133 (java) S 12123 12123 8601 0 -1 64 1 0 0 0 0 0 0 0 18 0 9 0 59318754 868974592 7116 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12134] ppid=12123 vsize=848608 CPUtime=0.33
/proc/12128/task/12134/stat : 12134 (java) S 12123 12123 8601 0 -1 64 1540 0 0 0 32 1 0 0 15 0 9 0 59318757 868974592 7116 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12135] ppid=12123 vsize=848608 CPUtime=0
/proc/12128/task/12135/stat : 12135 (java) S 12123 12123 8601 0 -1 64 2 0 0 0 0 0 0 0 20 0 9 0 59318758 868974592 7116 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12136] ppid=12123 vsize=848608 CPUtime=0
/proc/12128/task/12136/stat : 12136 (java) S 12123 12123 8601 0 -1 64 2 0 0 0 0 0 0 0 15 0 9 0 59318758 868974592 7116 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 1.47
Current children cumulated vsize (KiB) 853964

[startup+3.10276 s]
/proc/loadavg: 2.01 2.03 2.00 3/82 12136
/proc/meminfo: memFree=1084132/2055920 swapFree=4192812/4192956
[pid=12123] ppid=12121 vsize=5356 CPUtime=0
/proc/12123/stat : 12123 (concrete.sh) S 12121 12123 8601 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59318748 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/12123/statm: 1339 244 201 169 0 50 0
[pid=12128] ppid=12123 vsize=848608 CPUtime=3.06
/proc/12128/stat : 12128 (java) S 12123 12123 8601 0 -1 0 9623 0 1 0 299 7 0 0 18 0 9 0 59318749 868974592 8433 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 18446744073709551615 0 0 17 1 0 0
/proc/12128/statm: 212152 8433 2625 9 0 206630 0
[pid=12128/tid=12129] ppid=12123 vsize=848608 CPUtime=2.42
/proc/12128/task/12129/stat : 12129 (java) R 12123 12123 8601 0 -1 64 3946 0 1 0 239 3 0 0 25 0 9 0 59318750 868974592 8433 996147200 134512640 134550932 4294955408 18446744073709551615 4126476550 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12130] ppid=12123 vsize=848608 CPUtime=0.28
/proc/12128/task/12130/stat : 12130 (java) S 12123 12123 8601 0 -1 64 3435 0 0 0 27 1 0 0 16 0 9 0 59318751 868974592 8433 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12131] ppid=12123 vsize=848608 CPUtime=0
/proc/12128/task/12131/stat : 12131 (java) S 12123 12123 8601 0 -1 64 13 0 0 0 0 0 0 0 16 0 9 0 59318752 868974592 8433 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12132] ppid=12123 vsize=848608 CPUtime=0
/proc/12128/task/12132/stat : 12132 (java) S 12123 12123 8601 0 -1 64 4 0 0 0 0 0 0 0 15 0 9 0 59318752 868974592 8433 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12133] ppid=12123 vsize=848608 CPUtime=0
/proc/12128/task/12133/stat : 12133 (java) S 12123 12123 8601 0 -1 64 1 0 0 0 0 0 0 0 18 0 9 0 59318754 868974592 8433 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12134] ppid=12123 vsize=848608 CPUtime=0.33
/proc/12128/task/12134/stat : 12134 (java) S 12123 12123 8601 0 -1 64 1542 0 0 0 32 1 0 0 15 0 9 0 59318757 868974592 8433 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12135] ppid=12123 vsize=848608 CPUtime=0
/proc/12128/task/12135/stat : 12135 (java) S 12123 12123 8601 0 -1 64 2 0 0 0 0 0 0 0 20 0 9 0 59318758 868974592 8433 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12136] ppid=12123 vsize=848608 CPUtime=0
/proc/12128/task/12136/stat : 12136 (java) S 12123 12123 8601 0 -1 64 2 0 0 0 0 0 0 0 15 0 9 0 59318758 868974592 8433 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 3.06
Current children cumulated vsize (KiB) 853964

[startup+6.30212 s]
/proc/loadavg: 2.17 2.07 2.01 4/83 12137
/proc/meminfo: memFree=1069468/2055920 swapFree=4192812/4192956
[pid=12123] ppid=12121 vsize=5356 CPUtime=0
/proc/12123/stat : 12123 (concrete.sh) S 12121 12123 8601 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59318748 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/12123/statm: 1339 244 201 169 0 50 0
[pid=12128] ppid=12123 vsize=849740 CPUtime=6.22
/proc/12128/stat : 12128 (java) S 12123 12123 8601 0 -1 0 12092 0 1 0 614 8 0 0 18 0 10 0 59318749 870133760 10553 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 18446744073709551615 0 0 17 1 0 0
/proc/12128/statm: 212435 10553 2626 9 0 206913 0
[pid=12128/tid=12129] ppid=12123 vsize=849740 CPUtime=5.19
/proc/12128/task/12129/stat : 12129 (java) R 12123 12123 8601 0 -1 64 4086 0 1 0 516 3 0 0 25 0 10 0 59318750 870133760 10553 996147200 134512640 134550932 4294955408 18446744073709551615 4126480613 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12130] ppid=12123 vsize=849740 CPUtime=0.65
/proc/12128/task/12130/stat : 12130 (java) S 12123 12123 8601 0 -1 64 5748 0 0 0 63 2 0 0 16 0 10 0 59318751 870133760 10553 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12131] ppid=12123 vsize=849740 CPUtime=0
/proc/12128/task/12131/stat : 12131 (java) S 12123 12123 8601 0 -1 64 13 0 0 0 0 0 0 0 16 0 10 0 59318752 870133760 10553 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12132] ppid=12123 vsize=849740 CPUtime=0
/proc/12128/task/12132/stat : 12132 (java) S 12123 12123 8601 0 -1 64 4 0 0 0 0 0 0 0 16 0 10 0 59318752 870133760 10553 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12133] ppid=12123 vsize=849740 CPUtime=0
/proc/12128/task/12133/stat : 12133 (java) S 12123 12123 8601 0 -1 64 1 0 0 0 0 0 0 0 18 0 10 0 59318754 870133760 10553 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12134] ppid=12123 vsize=849740 CPUtime=0.35
/proc/12128/task/12134/stat : 12134 (java) S 12123 12123 8601 0 -1 64 1553 0 0 0 34 1 0 0 16 0 10 0 59318757 870133760 10553 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12135] ppid=12123 vsize=849740 CPUtime=0
/proc/12128/task/12135/stat : 12135 (java) S 12123 12123 8601 0 -1 64 2 0 0 0 0 0 0 0 20 0 10 0 59318758 870133760 10553 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12136] ppid=12123 vsize=849740 CPUtime=0
/proc/12128/task/12136/stat : 12136 (java) S 12123 12123 8601 0 -1 64 2 0 0 0 0 0 0 0 15 0 10 0 59318758 870133760 10553 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12137] ppid=12123 vsize=849740 CPUtime=0
/proc/12128/task/12137/stat : 12137 (java) S 12123 12123 8601 0 -1 64 5 0 0 0 0 0 0 0 25 0 10 0 59319230 870133760 10553 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 6.22
Current children cumulated vsize (KiB) 855096

[startup+12.7018 s]
/proc/loadavg: 2.16 2.07 2.01 3/83 12137
/proc/meminfo: memFree=1054428/2055920 swapFree=4192812/4192956
[pid=12123] ppid=12121 vsize=5356 CPUtime=0
/proc/12123/stat : 12123 (concrete.sh) S 12121 12123 8601 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59318748 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/12123/statm: 1339 244 201 169 0 50 0
[pid=12128] ppid=12123 vsize=852192 CPUtime=12.56
/proc/12128/stat : 12128 (java) S 12123 12123 8601 0 -1 0 15708 0 1 0 1241 15 0 0 18 0 10 0 59318749 872644608 13563 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 18446744073709551615 0 0 17 1 0 0
/proc/12128/statm: 213048 13563 2633 9 0 207526 0
[pid=12128/tid=12129] ppid=12123 vsize=852192 CPUtime=10.35
/proc/12128/task/12129/stat : 12129 (java) R 12123 12123 8601 0 -1 64 4538 0 1 0 1029 6 0 0 25 0 10 0 59318750 872644608 13563 996147200 134512640 134550932 4294955408 18446744073709551615 4127700001 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12130] ppid=12123 vsize=852192 CPUtime=1.21
/proc/12128/task/12130/stat : 12130 (java) S 12123 12123 8601 0 -1 64 7807 0 0 0 116 5 0 0 16 0 10 0 59318751 872644608 13563 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12131] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12131/stat : 12131 (java) S 12123 12123 8601 0 -1 64 13 0 0 0 0 0 0 0 16 0 10 0 59318752 872644608 13563 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12132] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12132/stat : 12132 (java) S 12123 12123 8601 0 -1 64 7 0 0 0 0 0 0 0 16 0 10 0 59318752 872644608 13563 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12133] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12133/stat : 12133 (java) S 12123 12123 8601 0 -1 64 1 0 0 0 0 0 0 0 18 0 10 0 59318754 872644608 13563 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12134] ppid=12123 vsize=852192 CPUtime=0.97
/proc/12128/task/12134/stat : 12134 (java) S 12123 12123 8601 0 -1 64 2655 0 0 0 95 2 0 0 15 0 10 0 59318757 872644608 13563 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12135] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12135/stat : 12135 (java) S 12123 12123 8601 0 -1 64 2 0 0 0 0 0 0 0 20 0 10 0 59318758 872644608 13563 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12136] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12136/stat : 12136 (java) S 12123 12123 8601 0 -1 64 2 0 0 0 0 0 0 0 15 0 10 0 59318758 872644608 13563 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12137] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12137/stat : 12137 (java) S 12123 12123 8601 0 -1 64 5 0 0 0 0 0 0 0 25 0 10 0 59319230 872644608 13563 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 12.56
Current children cumulated vsize (KiB) 857548

[startup+25.5023 s]
/proc/loadavg: 2.12 2.06 2.01 3/83 12137
/proc/meminfo: memFree=1002780/2055920 swapFree=4192812/4192956
[pid=12123] ppid=12121 vsize=5356 CPUtime=0
/proc/12123/stat : 12123 (concrete.sh) S 12121 12123 8601 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59318748 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/12123/statm: 1339 244 201 169 0 50 0
[pid=12128] ppid=12123 vsize=852192 CPUtime=25.22
/proc/12128/stat : 12128 (java) S 12123 12123 8601 0 -1 0 26548 0 1 0 2492 30 0 0 18 0 10 0 59318749 872644608 24360 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 18446744073709551615 0 0 17 1 0 0
/proc/12128/statm: 213048 24360 2633 9 0 207526 0
[pid=12128/tid=12129] ppid=12123 vsize=852192 CPUtime=21.04
/proc/12128/task/12129/stat : 12129 (java) R 12123 12123 8601 0 -1 64 5689 0 1 0 2090 14 0 0 25 0 10 0 59318750 872644608 24360 996147200 134512640 134550932 4294955408 18446744073709551615 4126914653 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12130] ppid=12123 vsize=852192 CPUtime=1.98
/proc/12128/task/12130/stat : 12130 (java) S 12123 12123 8601 0 -1 64 16958 0 0 0 189 9 0 0 16 0 10 0 59318751 872644608 24360 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12131] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12131/stat : 12131 (java) S 12123 12123 8601 0 -1 64 13 0 0 0 0 0 0 0 16 0 10 0 59318752 872644608 24360 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12132] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12132/stat : 12132 (java) S 12123 12123 8601 0 -1 64 8 0 0 0 0 0 0 0 16 0 10 0 59318752 872644608 24360 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12133] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12133/stat : 12133 (java) S 12123 12123 8601 0 -1 64 1 0 0 0 0 0 0 0 18 0 10 0 59318754 872644608 24360 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12134] ppid=12123 vsize=852192 CPUtime=2.17
/proc/12128/task/12134/stat : 12134 (java) S 12123 12123 8601 0 -1 64 3192 0 0 0 211 6 0 0 15 0 10 0 59318757 872644608 24360 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12135] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12135/stat : 12135 (java) S 12123 12123 8601 0 -1 64 2 0 0 0 0 0 0 0 20 0 10 0 59318758 872644608 24360 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12136] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12136/stat : 12136 (java) S 12123 12123 8601 0 -1 64 2 0 0 0 0 0 0 0 15 0 10 0 59318758 872644608 24360 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12137] ppid=12123 vsize=852192 CPUtime=0

################
# More data... #
################


Solver just ended. Dumping a history of the last processes samples

[startup+108.703 s]
/proc/loadavg: 2.14 2.09 2.02 3/82 12138
/proc/meminfo: memFree=578708/2055920 swapFree=4192812/4192956
[pid=12123] ppid=12121 vsize=5356 CPUtime=0
/proc/12123/stat : 12123 (concrete.sh) S 12121 12123 8601 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59318748 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/12123/statm: 1339 244 201 169 0 50 0
[pid=12128] ppid=12123 vsize=852192 CPUtime=107.63
/proc/12128/stat : 12128 (java) S 12123 12123 8601 0 -1 0 120949 0 1 0 10652 111 0 0 18 0 9 0 59318749 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 18446744073709551615 0 0 17 1 0 0
/proc/12128/statm: 213048 118685 2634 9 0 207526 0
[pid=12128/tid=12129] ppid=12123 vsize=852192 CPUtime=91.73
/proc/12128/task/12129/stat : 12129 (java) R 12123 12123 8601 0 -1 64 11863 0 1 0 9125 48 0 0 25 0 9 0 59318750 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4127972144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12130] ppid=12123 vsize=852192 CPUtime=8.97
/proc/12128/task/12130/stat : 12130 (java) S 12123 12123 8601 0 -1 64 104180 0 0 0 851 46 0 0 17 0 9 0 59318751 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12131] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12131/stat : 12131 (java) S 12123 12123 8601 0 -1 64 13 0 0 0 0 0 0 0 16 0 9 0 59318752 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12132] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12132/stat : 12132 (java) S 12123 12123 8601 0 -1 64 9 0 0 0 0 0 0 0 16 0 9 0 59318752 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12133] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12133/stat : 12133 (java) S 12123 12123 8601 0 -1 64 1 0 0 0 0 0 0 0 18 0 9 0 59318754 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12134] ppid=12123 vsize=852192 CPUtime=6.89
/proc/12128/task/12134/stat : 12134 (java) S 12123 12123 8601 0 -1 64 4194 0 0 0 674 15 0 0 15 0 9 0 59318757 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12135] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12135/stat : 12135 (java) S 12123 12123 8601 0 -1 64 2 0 0 0 0 0 0 0 20 0 9 0 59318758 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12136] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12136/stat : 12136 (java) S 12123 12123 8601 0 -1 64 2 0 0 0 0 0 0 0 15 0 9 0 59318758 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 107.63
Current children cumulated vsize (KiB) 857548

[startup+111.902 s]
/proc/loadavg: 2.13 2.08 2.01 3/82 12138
/proc/meminfo: memFree=576916/2055920 swapFree=4192812/4192956
[pid=12123] ppid=12121 vsize=5356 CPUtime=0
/proc/12123/stat : 12123 (concrete.sh) S 12121 12123 8601 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59318748 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/12123/statm: 1339 244 201 169 0 50 0
[pid=12128] ppid=12123 vsize=852192 CPUtime=110.8
/proc/12128/stat : 12128 (java) S 12123 12123 8601 0 -1 0 120950 0 1 0 10969 111 0 0 18 0 9 0 59318749 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 18446744073709551615 0 0 17 1 0 0
/proc/12128/statm: 213048 118685 2634 9 0 207526 0
[pid=12128/tid=12129] ppid=12123 vsize=852192 CPUtime=94.85
/proc/12128/task/12129/stat : 12129 (java) R 12123 12123 8601 0 -1 64 11864 0 1 0 9437 48 0 0 25 0 9 0 59318750 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4127721818 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12130] ppid=12123 vsize=852192 CPUtime=9.01
/proc/12128/task/12130/stat : 12130 (java) S 12123 12123 8601 0 -1 64 104180 0 0 0 855 46 0 0 16 0 9 0 59318751 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12131] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12131/stat : 12131 (java) S 12123 12123 8601 0 -1 64 13 0 0 0 0 0 0 0 16 0 9 0 59318752 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12132] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12132/stat : 12132 (java) S 12123 12123 8601 0 -1 64 9 0 0 0 0 0 0 0 16 0 9 0 59318752 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12133] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12133/stat : 12133 (java) S 12123 12123 8601 0 -1 64 1 0 0 0 0 0 0 0 18 0 9 0 59318754 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12134] ppid=12123 vsize=852192 CPUtime=6.9
/proc/12128/task/12134/stat : 12134 (java) S 12123 12123 8601 0 -1 64 4194 0 0 0 675 15 0 0 15 0 9 0 59318757 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12135] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12135/stat : 12135 (java) S 12123 12123 8601 0 -1 64 2 0 0 0 0 0 0 0 20 0 9 0 59318758 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12136] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12136/stat : 12136 (java) S 12123 12123 8601 0 -1 64 2 0 0 0 0 0 0 0 15 0 9 0 59318758 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 110.8
Current children cumulated vsize (KiB) 857548

[startup+112.702 s]
/proc/loadavg: 2.13 2.08 2.01 3/82 12138
/proc/meminfo: memFree=576404/2055920 swapFree=4192812/4192956
[pid=12123] ppid=12121 vsize=5356 CPUtime=0
/proc/12123/stat : 12123 (concrete.sh) S 12121 12123 8601 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59318748 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/12123/statm: 1339 244 201 169 0 50 0
[pid=12128] ppid=12123 vsize=852192 CPUtime=111.59
/proc/12128/stat : 12128 (java) S 12123 12123 8601 0 -1 0 120950 0 1 0 11048 111 0 0 18 0 9 0 59318749 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 18446744073709551615 0 0 17 1 0 0
/proc/12128/statm: 213048 118685 2634 9 0 207526 0
[pid=12128/tid=12129] ppid=12123 vsize=852192 CPUtime=95.64
/proc/12128/task/12129/stat : 12129 (java) R 12123 12123 8601 0 -1 64 11864 0 1 0 9516 48 0 0 25 0 9 0 59318750 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4127964222 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12130] ppid=12123 vsize=852192 CPUtime=9.01
/proc/12128/task/12130/stat : 12130 (java) S 12123 12123 8601 0 -1 64 104180 0 0 0 855 46 0 0 16 0 9 0 59318751 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12131] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12131/stat : 12131 (java) S 12123 12123 8601 0 -1 64 13 0 0 0 0 0 0 0 16 0 9 0 59318752 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12132] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12132/stat : 12132 (java) S 12123 12123 8601 0 -1 64 9 0 0 0 0 0 0 0 16 0 9 0 59318752 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12133] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12133/stat : 12133 (java) S 12123 12123 8601 0 -1 64 1 0 0 0 0 0 0 0 18 0 9 0 59318754 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12134] ppid=12123 vsize=852192 CPUtime=6.9
/proc/12128/task/12134/stat : 12134 (java) S 12123 12123 8601 0 -1 64 4194 0 0 0 675 15 0 0 15 0 9 0 59318757 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12135] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12135/stat : 12135 (java) S 12123 12123 8601 0 -1 64 2 0 0 0 0 0 0 0 20 0 9 0 59318758 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12136] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12136/stat : 12136 (java) S 12123 12123 8601 0 -1 64 2 0 0 0 0 0 0 0 15 0 9 0 59318758 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 111.59
Current children cumulated vsize (KiB) 857548

[startup+113.501 s]
/proc/loadavg: 2.13 2.08 2.01 4/82 12138
/proc/meminfo: memFree=575828/2055920 swapFree=4192812/4192956
[pid=12123] ppid=12121 vsize=5356 CPUtime=0
/proc/12123/stat : 12123 (concrete.sh) S 12121 12123 8601 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59318748 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/12123/statm: 1339 244 201 169 0 50 0
[pid=12128] ppid=12123 vsize=852192 CPUtime=112.38
/proc/12128/stat : 12128 (java) S 12123 12123 8601 0 -1 0 120953 0 1 0 11127 111 0 0 18 0 9 0 59318749 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 18446744073709551615 0 0 17 1 0 0
/proc/12128/statm: 213048 118685 2634 9 0 207526 0
[pid=12128/tid=12129] ppid=12123 vsize=852192 CPUtime=96.3
/proc/12128/task/12129/stat : 12129 (java) R 12123 12123 8601 0 -1 64 11867 0 1 0 9582 48 0 0 25 0 9 0 59318750 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12130] ppid=12123 vsize=852192 CPUtime=9.14
/proc/12128/task/12130/stat : 12130 (java) R 12123 12123 8601 0 -1 64 104180 0 0 0 868 46 0 0 16 0 9 0 59318751 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 102549491 0 0 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12131] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12131/stat : 12131 (java) S 12123 12123 8601 0 -1 64 13 0 0 0 0 0 0 0 16 0 9 0 59318752 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12132] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12132/stat : 12132 (java) S 12123 12123 8601 0 -1 64 9 0 0 0 0 0 0 0 16 0 9 0 59318752 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12133] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12133/stat : 12133 (java) S 12123 12123 8601 0 -1 64 1 0 0 0 0 0 0 0 18 0 9 0 59318754 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12134] ppid=12123 vsize=852192 CPUtime=6.9
/proc/12128/task/12134/stat : 12134 (java) S 12123 12123 8601 0 -1 64 4194 0 0 0 675 15 0 0 16 0 9 0 59318757 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12135] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12135/stat : 12135 (java) S 12123 12123 8601 0 -1 64 2 0 0 0 0 0 0 0 20 0 9 0 59318758 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12136] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12136/stat : 12136 (java) S 12123 12123 8601 0 -1 64 2 0 0 0 0 0 0 0 15 0 9 0 59318758 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 112.38
Current children cumulated vsize (KiB) 857548

[startup+113.704 s]
/proc/loadavg: 2.13 2.08 2.01 4/82 12138
/proc/meminfo: memFree=575828/2055920 swapFree=4192812/4192956
[pid=12123] ppid=12121 vsize=5356 CPUtime=0
/proc/12123/stat : 12123 (concrete.sh) S 12121 12123 8601 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59318748 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/12123/statm: 1339 244 201 169 0 50 0
[pid=12128] ppid=12123 vsize=852192 CPUtime=112.58
/proc/12128/stat : 12128 (java) S 12123 12123 8601 0 -1 0 120954 0 1 0 11147 111 0 0 18 0 9 0 59318749 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 18446744073709551615 0 0 17 1 0 0
/proc/12128/statm: 213048 118685 2634 9 0 207526 0
[pid=12128/tid=12129] ppid=12123 vsize=852192 CPUtime=96.49
/proc/12128/task/12129/stat : 12129 (java) R 12123 12123 8601 0 -1 64 11868 0 1 0 9601 48 0 0 25 0 9 0 59318750 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12130] ppid=12123 vsize=852192 CPUtime=9.15
/proc/12128/task/12130/stat : 12130 (java) R 12123 12123 8601 0 -1 64 104180 0 0 0 869 46 0 0 15 0 9 0 59318751 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 102546970 0 0 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12131] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12131/stat : 12131 (java) S 12123 12123 8601 0 -1 64 13 0 0 0 0 0 0 0 16 0 9 0 59318752 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12132] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12132/stat : 12132 (java) S 12123 12123 8601 0 -1 64 9 0 0 0 0 0 0 0 16 0 9 0 59318752 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12133] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12133/stat : 12133 (java) S 12123 12123 8601 0 -1 64 1 0 0 0 0 0 0 0 18 0 9 0 59318754 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12134] ppid=12123 vsize=852192 CPUtime=6.9
/proc/12128/task/12134/stat : 12134 (java) S 12123 12123 8601 0 -1 64 4194 0 0 0 675 15 0 0 15 0 9 0 59318757 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12135] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12135/stat : 12135 (java) S 12123 12123 8601 0 -1 64 2 0 0 0 0 0 0 0 20 0 9 0 59318758 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12136] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12136/stat : 12136 (java) S 12123 12123 8601 0 -1 64 2 0 0 0 0 0 0 0 15 0 9 0 59318758 872644608 118685 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 112.58
Current children cumulated vsize (KiB) 857548

[startup+113.802 s]
/proc/loadavg: 2.13 2.08 2.01 4/82 12138
/proc/meminfo: memFree=575828/2055920 swapFree=4192812/4192956
[pid=12123] ppid=12121 vsize=5356 CPUtime=0
/proc/12123/stat : 12123 (concrete.sh) S 12121 12123 8601 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59318748 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/12123/statm: 1339 244 201 169 0 50 0
[pid=12128] ppid=12123 vsize=852192 CPUtime=112.65
/proc/12128/stat : 12128 (java) S 12123 12123 8601 0 -1 0 120959 0 1 0 11154 111 0 0 18 0 9 0 59318749 872644608 118687 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 18446744073709551615 0 0 17 1 0 0
/proc/12128/statm: 213048 118687 2634 9 0 207526 0
[pid=12128/tid=12129] ppid=12123 vsize=852192 CPUtime=96.52
/proc/12128/task/12129/stat : 12129 (java) S 12123 12123 8601 0 -1 64 11873 0 1 0 9604 48 0 0 25 0 9 0 59318750 872644608 118687 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12130] ppid=12123 vsize=852192 CPUtime=9.19
/proc/12128/task/12130/stat : 12130 (java) S 12123 12123 8601 0 -1 64 104180 0 0 0 873 46 0 0 16 0 9 0 59318751 872644608 118687 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12131] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12131/stat : 12131 (java) S 12123 12123 8601 0 -1 64 13 0 0 0 0 0 0 0 16 0 9 0 59318752 872644608 118687 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12132] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12132/stat : 12132 (java) S 12123 12123 8601 0 -1 64 9 0 0 0 0 0 0 0 16 0 9 0 59318752 872644608 118687 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12133] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12133/stat : 12133 (java) S 12123 12123 8601 0 -1 64 1 0 0 0 0 0 0 0 18 0 9 0 59318754 872644608 118687 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12134] ppid=12123 vsize=852192 CPUtime=6.9
/proc/12128/task/12134/stat : 12134 (java) S 12123 12123 8601 0 -1 64 4194 0 0 0 675 15 0 0 15 0 9 0 59318757 872644608 118687 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12135] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12135/stat : 12135 (java) S 12123 12123 8601 0 -1 64 2 0 0 0 0 0 0 0 20 0 9 0 59318758 872644608 118687 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=12128/tid=12136] ppid=12123 vsize=852192 CPUtime=0
/proc/12128/task/12136/stat : 12136 (java) S 12123 12123 8601 0 -1 64 2 0 0 0 0 0 0 0 15 0 9 0 59318758 872644608 118687 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 112.65
Current children cumulated vsize (KiB) 857548

Child status: 0
Real time (s): 113.881
CPU time (s): 112.744
CPU user time (s): 111.549
CPU system time (s): 1.19482
CPU usage (%): 99.0017
Max. virtual memory (cumulated for all children) (KiB): 858196

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 111.549
system time used= 1.19482
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 121604
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 32839
involuntary context switches= 34187

runsolver used 0.156976 second user time and 0.412937 second system time

The end

Launcher Data 

Begin job on node9 at 2009-07-16 07:35:16
IDJOB=2083134
IDBENCH=57715
IDSOLVER=735
FILE ID=node9/2083134-1247722516
PBS_JOBID= 9521365
Free space on /tmp= 66372 MiB

SOLVER NAME= Concrete 2009-06-10
BENCH NAME= CPAI08/csp/fapp/fapp10/normalized-fapp10-0900-9.xml
COMMAND LINE= HOME/concrete.sh -c BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-2083134-1247722516/watcher-2083134-1247722516 -o /tmp/evaluation-result-2083134-1247722516/solver-2083134-1247722516 -C 1800 -W 2000 -M 900  HOME/concrete.sh -c HOME/instance-2083134-1247722516.xml

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 900 MiB
MAX NB THREAD= 0

MD5SUM BENCH= bb8fed9f370e164b93c37276047ba0d0
RANDOM SEED=1599678663

node9.alineos.net Linux 2.6.9-22.EL.rootsmp #1 SMP Mon Oct 3 08:59:52 CEST 2005

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.216
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5914.62
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.216
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5586.94
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:      2055920 kB
MemFree:       1106580 kB
Buffers:         75700 kB
Cached:         280416 kB
SwapCached:          0 kB
Active:         601928 kB
Inactive:       276916 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      2055920 kB
LowFree:       1106580 kB
SwapTotal:     4192956 kB
SwapFree:      4192812 kB
Dirty:            2784 kB
Writeback:           0 kB
Mapped:         534244 kB
Slab:            54924 kB
Committed_AS:  1052316 kB
PageTables:       2832 kB
VmallocTotal: 536870911 kB
VmallocUsed:    264952 kB
VmallocChunk: 536605679 kB
HugePages_Total:     0
HugePages_Free:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 66372 MiB
End job on node9 at 2009-07-16 07:37:10