Trace number 2083112

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Concrete 2009-07-14SAT 53.5779 55.0958

DiagnosticValue
CHECKS2143260
NODES722

General information on the benchmark

Namecsp/fapp/fapp08/
normalized-fapp08-0700-8.xml
MD5SUM76f5c8323181a5b7c43a501cf4eaddc1
Bench Category2-ARY-INT (binary constraints in intension)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark33.9748
Satisfiable
(Un)Satisfiability was proved
Number of variables700
Number of constraints7122
Maximum constraint arity2
Maximum domain size282
Number of constraints which are defined in extension0
Number of constraints which are defined in intension7122
Global constraints used (with number of constraints)

Solver Data 

45.95/46.78	c CPU-LOAD 36.73
45.95/46.78	c LOADED 700 V (282 E), 3834 ExtensionConstraint2D, arity 2, 36.73 cpu
53.41/54.99	v 2452 -2608 -2408 2456 -2680 2300 -2532 2468 -2360 -2276 -2676 -2624 -2256 2412 2300 -2424 2348 -2632 2624 2492 -2620 2596 -2720 -2232 2672 2708 -2452 2500 -2576 2568 -2436 2472 -2300 -2492 -2360 2360 2224 -2304 -2312 2308 -2628 -2544 -2484 -2272 -2424 -2356 -2412 -2712 -2396 2160 -2388 -2508 -2588 -2240 2160 2624 2380 -2228 2488 2408 -2200 -2704 2380 2532 -2716 -2700 -2428 2532 2476 2520 -2636 -2184 2472 2160 2236 2636 2408 -2360 -2696 -2332 -2624 -2420 -2308 -2616 2396 -2160 -2348 -2564 2280 -2404 2680 -2584 -2604 -2720 2512 -2348 -2372 -2224 -2688 -2660 -2236 -2720 -2532 -2296 2200 2532 -2200 -2676 2180 -2672 2232 2716 -2264 -2208 2520 -2396 2364 2272 2520 -2220 -2616 2400 -2456 -2528 2352 2384 2396 -2588 2216 -2224 2472 2524 -2668 2332 -2252 2344 -2488 -2568 -2160 -2260 -2712 -2164 2268 -2212 -2392 -2488 -2560 -2560 -2364 -2636 -2664 2400 2656 2276 -2372 -2416 2372 -2708 -2708 2408 2672 2520 2528 -2608 -2440 2632 -2636 2324 -2620 2276 2160 -2636 -2480 -2180 -2600 -2608 -2680 -2544 -2680 -2640 2632 -2544 -2352 -2284 2520 -2656 -2464 -2384 -2188 2160 -2440 -2388 -2376 -2508 2648 -2484 -2520 2568 2292 -2664 2680 -2340 -2436 -2388 2604 -2440 -2508 -2696 -2504 -2584 -2400 2380 -2600 -2292 -2500 -2292 -2192 2208 -2480 -2588 -2208 2232 2304 -2432 2596 -2584 2352 2440 -2484 -2268 -2352 -2628 -2256 -2416 2228 -2632 -2496 -2560 -2720 -2428 -2680 -2308 2628 -2624 -2580 -2392 2332 -2708 2368 -2192 -2324 -2224 -2680 2276 -2696 2400 2648 -2356 2452 -2400 -2616 -2524 -2432 -2548 2316 2616 2160 2592 -2284 -2400 2372 -2412 2216 -2608 -2256 -2412 -2584 -2340 -2244 -2720 -2524 -2560 2660 2636 -2428 -2312 -2184 2360 2464 2456 -2568 -2360 -2676 2212 -2560 -2656 -2452 2512 2424 2316 -2664 2416 -2680 2652 -2584 2160 2680 -2532 -2532 2584 -2596 2632 -2476 -2368 -2292 -2720 -2420 -2488 2472 -2272 2672 2268 -2288 -2420 -2672 -2200 -2376 -2488 2452 -2644 -2444 2412 -2680 -2292 2304 -2544 -2600 -2680 -2200 -2568 -2200 -2268 -2332 -2520 -2308 -2300 2308 -2616 2176 -2628 -2236 -2624 -2528 2276 -2200 -2184 2644 -2176 -2648 2496 -2200 2200 -2628 2200 -2448 -2584 2548 -2668 2336 -2172 -2584 -2184 -2460 -2680 -2488 -2680 2568 -2200 -2308 -2588 -2468 2520 -2648 -2720 -2444 -2284 2564 -2380 2276 -2624 -2700 -2420 -2448 2400 -2668 -2480 -2504 -2568 -2476 2236 -2372 2544 -2616 -2692 -2668 -2356 2360 -2272 -2628 -2664 -2412 -2460 -2696 -2336 2200 2496 2376 2528 -2520 -2512 -2624 -2312 -2200 -2448 2416 -2624 -2644 -2588 2560 -2396 2204 -2328 2628 -2220 2448 2580 -2528 -2460 -2200 -2328 2528 2168 -2676 -2356 -2644 -2356 -2352 2468 2552 -2408 2348 2668 -2584 -2240 -2296 2252 -2624 -2448 -2580 2624 -2400 -2576 -2488 -2408 2268 -2348 -2388 2676 -2720 -2420 -2568 -2160 2160 -2684 2320 -2332 -2716 -2368 -2452 -2160 -2436 -2316 -2552 2700 -2620 -2608 -2572 2672 -2252 2392 2232 2160 -2196 -2436 -2532 -2664 -2368 -2200 2168 2256 -2360 -2548 -2316 -2540 -2224 -2184 2412 -2160 -2676 -2344 -2608 2632 2332 -2552 -2320 2616 -2380 -2376 2452 -2548 -2532 2160 -2508 -2308 -2532 -2588 -2704 -2564 2320 -2276 -2620 -2200 -2388 2200 -2572 -2396 2328 2176 -2640 -2588 2696 2392 -2676 2664 -2396 -2372 -2392 -2668 2316 -2656 -2584 2436 -2672 2572 -2492 -2476 -2584 -2420 2448 -2572 -2720 -2576 2308 -2352 -2428 -2196 -2428 2164 -2720 -2584 -2604 -2560 -2544 -2356 -2340 2448 -2640 -2492 -2592 -2220 2628 2392 2248 -2588 2560 2536 -2628 2300 -2656 -2624 -2584 -2492 2320 2344 -2372 -2160 -2216 -2316 -2200 2180 -2536 -2344 2172 -2608 -2312 -2668 -2440 2400 -2604 2160 2376 -2428 2492 -2436 2492 -2448 2376 -2720 2520 -2304 -2636 -2680 2676 -2432 -2680 2448 -2680 -2216 -2476 -2720 2160 2200 2428 -2388 2528 -2720 -2528 2324 -2504 2656 2408 2544 2680 -2456 2340 2284 -2200 -2388 2308 -2404 -2628 -2336 -2388 -2348 -2200 -2660 2300 -2432 -2392 -2548 2512 -2524 -2716 -2576 -2656 -2572 -2200 -2384 -2508 -2428 -2332 -2568 2160 -2260 -2656 -2660 -2216 -2336 -2560 2532 -2420 2720 -2392 -2624 -2608 -2436 -2352 2160 -2336 -2324 -2664 2248 -2412 -2216 -2204 -2656 -2592 -2348 2320 -2640 
53.41/55.00	s SATISFIABLE
53.41/55.00	d NODES 722
53.41/55.00	d CHECKS 2143257
53.41/55.00	c heuristic-cpu: 0.0
53.41/55.00	c prepro-constraint-ccks: 0
53.41/55.00	c prepro-constraint-presenceccks: 0
53.41/55.00	c prepro-cpu: 5.3399997
53.41/55.00	c prepro-matrix2d-ccks: 386178
53.41/55.00	c prepro-matrix2d-presenceccks: 236634
53.41/55.00	c prepro-removed: 31661
53.41/55.00	c search-cpu: 1.3100004
53.41/55.00	c search-nps: 551.14484
53.41/55.00	c total-assgn: 722
53.41/55.00	c total-constraint-ccks: 0
53.41/55.00	c total-constraint-presence-ccks: 0
53.41/55.00	c total-cpu: 6.9900002
53.41/55.00	c total-matrix2d-ccks: 2143257
53.41/55.00	c total-matrix2d-presence-ccks: 2154655
53.41/55.00	c TOTAL-NODES 722
53.41/55.00	c TOTAL-CPU 6.9900002

Verifier Data 

OK

Watcher Data 

runsolver version 3.2.9 (svn:492) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-2083112-1247722479/watcher-2083112-1247722479 -o /tmp/evaluation-result-2083112-1247722479/solver-2083112-1247722479 -C 1800 -W 2000 -M 900 HOME/concrete.sh -c HOME/instance-2083112-1247722479.xml 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 921600 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 972800 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.00 2.00 2.00 4/81 21526
/proc/meminfo: memFree=1630352/2055920 swapFree=4192812/4192956
[pid=21526] ppid=21524 vsize=5260 CPUtime=0
/proc/21526/stat : 21526 (concrete.sh) R 21524 21526 19964 0 -1 4194304 66 0 0 0 0 0 0 0 20 0 1 0 59314536 5386240 50 996147200 4194304 4889804 548682068816 18446744073709551615 217868980019 0 0 4096 0 0 0 0 17 1 0 0
/proc/21526/statm: 1315 50 37 169 0 14 0

[startup+0.0674129 s]
/proc/loadavg: 2.00 2.00 2.00 4/81 21526
/proc/meminfo: memFree=1630352/2055920 swapFree=4192812/4192956
[pid=21526] ppid=21524 vsize=5356 CPUtime=0
/proc/21526/stat : 21526 (concrete.sh) S 21524 21526 19964 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59314536 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 217871606596 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21526/statm: 1339 244 201 169 0 50 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 5356

[startup+0.101421 s]
/proc/loadavg: 2.00 2.00 2.00 4/81 21526
/proc/meminfo: memFree=1630352/2055920 swapFree=4192812/4192956
[pid=21526] ppid=21524 vsize=5356 CPUtime=0
/proc/21526/stat : 21526 (concrete.sh) S 21524 21526 19964 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59314536 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 217871606596 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21526/statm: 1339 244 201 169 0 50 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 5356

[startup+0.301444 s]
/proc/loadavg: 2.00 2.00 2.00 4/81 21526
/proc/meminfo: memFree=1630352/2055920 swapFree=4192812/4192956
[pid=21526] ppid=21524 vsize=5356 CPUtime=0
/proc/21526/stat : 21526 (concrete.sh) S 21524 21526 19964 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59314536 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 217871606596 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21526/statm: 1339 244 201 169 0 50 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 5356

[startup+0.7013 s]
/proc/loadavg: 2.00 2.00 2.00 4/81 21526
/proc/meminfo: memFree=1630352/2055920 swapFree=4192812/4192956
[pid=21526] ppid=21524 vsize=5356 CPUtime=0
/proc/21526/stat : 21526 (concrete.sh) S 21524 21526 19964 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59314536 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 217871606596 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21526/statm: 1339 244 201 169 0 50 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 5356

[startup+1.50239 s]
/proc/loadavg: 2.00 2.00 2.00 3/91 21539
/proc/meminfo: memFree=1619200/2055920 swapFree=4192812/4192956
[pid=21526] ppid=21524 vsize=5356 CPUtime=0
/proc/21526/stat : 21526 (concrete.sh) S 21524 21526 19964 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59314536 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 217871606596 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21526/statm: 1339 244 201 169 0 50 0
[pid=21531] ppid=21526 vsize=847456 CPUtime=0.84
/proc/21531/stat : 21531 (java) S 21526 21526 19964 0 -1 0 5560 0 1 0 73 11 0 0 18 0 9 0 59314537 867794944 4859 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 18446744073709551615 0 0 17 1 0 0
/proc/21531/statm: 211864 4859 1912 9 0 206349 0
[pid=21531/tid=21532] ppid=21526 vsize=847456 CPUtime=0.56
/proc/21531/task/21532/stat : 21532 (java) R 21526 21526 19964 0 -1 64 3446 0 1 0 53 3 0 0 18 0 9 0 59314538 867794944 4859 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21533] ppid=21526 vsize=847456 CPUtime=0.01
/proc/21531/task/21533/stat : 21533 (java) R 21526 21526 19964 0 -1 64 194 0 0 0 1 0 0 0 16 0 9 0 59314539 867794944 4859 996147200 134512640 134550932 4294955408 18446744073709551615 103761630 0 0 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21534] ppid=21526 vsize=847456 CPUtime=0
/proc/21531/task/21534/stat : 21534 (java) S 21526 21526 19964 0 -1 64 13 0 0 0 0 0 0 0 16 0 9 0 59314540 867794944 4859 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21535] ppid=21526 vsize=847456 CPUtime=0
/proc/21531/task/21535/stat : 21535 (java) S 21526 21526 19964 0 -1 64 4 0 0 0 0 0 0 0 16 0 9 0 59314540 867794944 4859 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21536] ppid=21526 vsize=847456 CPUtime=0
/proc/21531/task/21536/stat : 21536 (java) S 21526 21526 19964 0 -1 64 0 0 0 0 0 0 0 0 18 0 9 0 59314542 867794944 4859 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21537] ppid=21526 vsize=847456 CPUtime=0.15
/proc/21531/task/21537/stat : 21537 (java) S 21526 21526 19964 0 -1 64 1223 0 0 0 15 0 0 0 16 0 9 0 59314546 867794944 4859 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21538] ppid=21526 vsize=847456 CPUtime=0
/proc/21531/task/21538/stat : 21538 (java) S 21526 21526 19964 0 -1 64 1 0 0 0 0 0 0 0 19 0 9 0 59314546 867794944 4859 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21539] ppid=21526 vsize=847456 CPUtime=0.08
/proc/21531/task/21539/stat : 21539 (java) S 21526 21526 19964 0 -1 64 1 0 0 0 2 6 0 0 15 0 9 0 59314546 867794944 4859 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 0.84
Current children cumulated vsize (KiB) 852812

[startup+3.10258 s]
/proc/loadavg: 2.00 2.00 2.00 3/91 21539
/proc/meminfo: memFree=1612800/2055920 swapFree=4192812/4192956
[pid=21526] ppid=21524 vsize=5356 CPUtime=0
/proc/21526/stat : 21526 (concrete.sh) S 21524 21526 19964 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59314536 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 217871606596 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21526/statm: 1339 244 201 169 0 50 0
[pid=21531] ppid=21526 vsize=847692 CPUtime=2.44
/proc/21531/stat : 21531 (java) S 21526 21526 19964 0 -1 0 8703 0 1 0 230 14 0 0 18 0 9 0 59314537 868036608 7748 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 18446744073709551615 0 0 17 1 0 0
/proc/21531/statm: 211923 7748 2625 9 0 206401 0
[pid=21531/tid=21532] ppid=21526 vsize=847692 CPUtime=1.74
/proc/21531/task/21532/stat : 21532 (java) R 21526 21526 19964 0 -1 64 3917 0 1 0 170 4 0 0 25 0 9 0 59314538 868036608 7748 996147200 134512640 134550932 4294955408 18446744073709551615 4126206112 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21533] ppid=21526 vsize=847692 CPUtime=0.24
/proc/21531/task/21533/stat : 21533 (java) S 21526 21526 19964 0 -1 64 2731 0 0 0 23 1 0 0 16 0 9 0 59314539 868036608 7748 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21534] ppid=21526 vsize=847692 CPUtime=0
/proc/21531/task/21534/stat : 21534 (java) S 21526 21526 19964 0 -1 64 13 0 0 0 0 0 0 0 16 0 9 0 59314540 868036608 7748 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21535] ppid=21526 vsize=847692 CPUtime=0
/proc/21531/task/21535/stat : 21535 (java) S 21526 21526 19964 0 -1 64 4 0 0 0 0 0 0 0 15 0 9 0 59314540 868036608 7748 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21536] ppid=21526 vsize=847692 CPUtime=0
/proc/21531/task/21536/stat : 21536 (java) S 21526 21526 19964 0 -1 64 0 0 0 0 0 0 0 0 18 0 9 0 59314542 868036608 7748 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21537] ppid=21526 vsize=847692 CPUtime=0.34
/proc/21531/task/21537/stat : 21537 (java) S 21526 21526 19964 0 -1 64 1358 0 0 0 34 0 0 0 15 0 9 0 59314546 868036608 7748 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21538] ppid=21526 vsize=847692 CPUtime=0
/proc/21531/task/21538/stat : 21538 (java) S 21526 21526 19964 0 -1 64 1 0 0 0 0 0 0 0 19 0 9 0 59314546 868036608 7748 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21539] ppid=21526 vsize=847692 CPUtime=0.08
/proc/21531/task/21539/stat : 21539 (java) S 21526 21526 19964 0 -1 64 1 0 0 0 2 6 0 0 15 0 9 0 59314546 868036608 7748 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 2.44
Current children cumulated vsize (KiB) 853048
heavy processes:

[startup+6.30195 s]
/proc/loadavg: 2.00 2.00 2.00 3/92 21540
/proc/meminfo: memFree=1597688/2055920 swapFree=4192812/4192956
[pid=21526] ppid=21524 vsize=5356 CPUtime=0
/proc/21526/stat : 21526 (concrete.sh) S 21524 21526 19964 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59314536 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 217871606596 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21526/statm: 1339 244 201 169 0 50 0
[pid=21531] ppid=21526 vsize=851072 CPUtime=5.6
/proc/21531/stat : 21531 (java) S 21526 21526 19964 0 -1 0 12270 0 1 0 541 19 0 0 18 0 10 0 59314537 871497728 11071 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 18446744073709551615 0 0 17 1 0 0
/proc/21531/statm: 212768 11071 2627 9 0 207246 0
[pid=21531/tid=21532] ppid=21526 vsize=851072 CPUtime=4.12
/proc/21531/task/21532/stat : 21532 (java) R 21526 21526 19964 0 -1 64 4123 0 1 0 407 5 0 0 25 0 10 0 59314538 871497728 11071 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21533] ppid=21526 vsize=851072 CPUtime=0.63
/proc/21531/task/21533/stat : 21533 (java) R 21526 21526 19964 0 -1 64 5163 0 0 0 59 4 0 0 16 0 10 0 59314539 871497728 11071 996147200 134512640 134550932 4294955408 18446744073709551615 104188790 0 0 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21534] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21534/stat : 21534 (java) S 21526 21526 19964 0 -1 64 13 0 0 0 0 0 0 0 16 0 10 0 59314540 871497728 11071 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21535] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21535/stat : 21535 (java) S 21526 21526 19964 0 -1 64 4 0 0 0 0 0 0 0 16 0 10 0 59314540 871497728 11071 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21536] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21536/stat : 21536 (java) S 21526 21526 19964 0 -1 64 0 0 0 0 0 0 0 0 18 0 10 0 59314542 871497728 11071 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21537] ppid=21526 vsize=851072 CPUtime=0.72
/proc/21531/task/21537/stat : 21537 (java) S 21526 21526 19964 0 -1 64 2283 0 0 0 70 2 0 0 15 0 10 0 59314546 871497728 11071 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21538] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21538/stat : 21538 (java) S 21526 21526 19964 0 -1 64 1 0 0 0 0 0 0 0 19 0 10 0 59314546 871497728 11071 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21539] ppid=21526 vsize=851072 CPUtime=0.08
/proc/21531/task/21539/stat : 21539 (java) S 21526 21526 19964 0 -1 64 1 0 0 0 2 6 0 0 15 0 10 0 59314546 871497728 11071 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21540] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21540/stat : 21540 (java) S 21526 21526 19964 0 -1 64 4 0 0 0 0 0 0 0 25 0 10 0 59314894 871497728 11071 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 5.6
Current children cumulated vsize (KiB) 856428

[startup+12.7027 s]
/proc/loadavg: 2.00 2.00 2.00 3/92 21540
/proc/meminfo: memFree=1583224/2055920 swapFree=4192812/4192956
[pid=21526] ppid=21524 vsize=5356 CPUtime=0
/proc/21526/stat : 21526 (concrete.sh) S 21524 21526 19964 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59314536 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 217871606596 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21526/statm: 1339 244 201 169 0 50 0
[pid=21531] ppid=21526 vsize=851072 CPUtime=12
/proc/21531/stat : 21531 (java) S 21526 21526 19964 0 -1 0 16062 0 1 0 1171 29 0 0 18 0 10 0 59314537 871497728 14715 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 18446744073709551615 0 0 17 1 0 0
/proc/21531/statm: 212768 14715 2633 9 0 207246 0
[pid=21531/tid=21532] ppid=21526 vsize=851072 CPUtime=9.29
/proc/21531/task/21532/stat : 21532 (java) R 21526 21526 19964 0 -1 64 4773 0 1 0 918 11 0 0 25 0 10 0 59314538 871497728 14715 996147200 134512640 134550932 4294955408 18446744073709551615 4128273817 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21533] ppid=21526 vsize=851072 CPUtime=1.14
/proc/21531/task/21533/stat : 21533 (java) S 21526 21526 19964 0 -1 64 7964 0 0 0 108 6 0 0 16 0 10 0 59314539 871497728 14715 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21534] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21534/stat : 21534 (java) S 21526 21526 19964 0 -1 64 13 0 0 0 0 0 0 0 16 0 10 0 59314540 871497728 14715 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21535] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21535/stat : 21535 (java) S 21526 21526 19964 0 -1 64 7 0 0 0 0 0 0 0 16 0 10 0 59314540 871497728 14715 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21536] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21536/stat : 21536 (java) S 21526 21526 19964 0 -1 64 0 0 0 0 0 0 0 0 18 0 10 0 59314542 871497728 14715 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21537] ppid=21526 vsize=851072 CPUtime=1.44
/proc/21531/task/21537/stat : 21537 (java) S 21526 21526 19964 0 -1 64 2621 0 0 0 141 3 0 0 15 0 10 0 59314546 871497728 14715 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21538] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21538/stat : 21538 (java) S 21526 21526 19964 0 -1 64 1 0 0 0 0 0 0 0 19 0 10 0 59314546 871497728 14715 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21539] ppid=21526 vsize=851072 CPUtime=0.08
/proc/21531/task/21539/stat : 21539 (java) S 21526 21526 19964 0 -1 64 1 0 0 0 2 6 0 0 15 0 10 0 59314546 871497728 14715 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21540] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21540/stat : 21540 (java) S 21526 21526 19964 0 -1 64 4 0 0 0 0 0 0 0 25 0 10 0 59314894 871497728 14715 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 12
Current children cumulated vsize (KiB) 856428

[startup+25.5012 s]
/proc/loadavg: 2.00 2.00 2.00 3/92 21540
/proc/meminfo: memFree=1534520/2055920 swapFree=4192812/4192956
[pid=21526] ppid=21524 vsize=5356 CPUtime=0
/proc/21526/stat : 21526 (concrete.sh) S 21524 21526 19964 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59314536 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 217871606596 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21526/statm: 1339 244 201 169 0 50 0
[pid=21531] ppid=21526 vsize=851072 CPUtime=24.77
/proc/21531/stat : 21531 (java) S 21526 21526 19964 0 -1 0 28076 0 1 0 2426 51 0 0 18 0 10 0 59314537 871497728 26694 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 18446744073709551615 0 0 17 1 0 0
/proc/21531/statm: 212768 26694 2633 9 0 207246 0
[pid=21531/tid=21532] ppid=21526 vsize=851072 CPUtime=19.73
/proc/21531/task/21532/stat : 21532 (java) R 21526 21526 19964 0 -1 64 6029 0 1 0 1950 23 0 0 25 0 10 0 59314538 871497728 26694 996147200 134512640 134550932 4294955408 18446744073709551615 4125351251 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21533] ppid=21526 vsize=851072 CPUtime=2.01
/proc/21531/task/21533/stat : 21533 (java) S 21526 21526 19964 0 -1 64 18256 0 0 0 189 12 0 0 16 0 10 0 59314539 871497728 26694 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21534] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21534/stat : 21534 (java) S 21526 21526 19964 0 -1 64 13 0 0 0 0 0 0 0 16 0 10 0 59314540 871497728 26694 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21535] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21535/stat : 21535 (java) S 21526 21526 19964 0 -1 64 7 0 0 0 0 0 0 0 16 0 10 0 59314540 871497728 26694 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21536] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21536/stat : 21536 (java) S 21526 21526 19964 0 -1 64 0 0 0 0 0 0 0 0 18 0 10 0 59314542 871497728 26694 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21537] ppid=21526 vsize=851072 CPUtime=2.91
/proc/21531/task/21537/stat : 21537 (java) S 21526 21526 19964 0 -1 64 3087 0 0 0 283 8 0 0 15 0 10 0 59314546 871497728 26694 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21538] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21538/stat : 21538 (java) S 21526 21526 19964 0 -1 64 1 0 0 0 0 0 0 0 19 0 10 0 59314546 871497728 26694 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21539] ppid=21526 vsize=851072 CPUtime=0.08
/proc/21531/task/21539/stat : 21539 (java) S 21526 21526 19964 0 -1 64 1 0 0 0 2 6 0 0 15 0 10 0 59314546 871497728 26694 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21540] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21540/stat : 21540 (java) S 21526 21526 19964 0 -1 64 4 0 0 0 0 0 0 0 25 0 10 0 59314894 871497728 26694 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 24.77

################
# More data... #
################

/proc/21526/statm: 1339 244 201 169 0 50 0
[pid=21531] ppid=21526 vsize=851072 CPUtime=50.35
/proc/21531/stat : 21531 (java) S 21526 21526 19964 0 -1 0 65974 0 1 0 4947 88 0 0 18 0 10 0 59314537 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 18446744073709551615 0 0 17 1 0 0
/proc/21531/statm: 212768 64556 2634 9 0 207246 0
[pid=21531/tid=21532] ppid=21526 vsize=851072 CPUtime=40.86
/proc/21531/task/21532/stat : 21532 (java) R 21526 21526 19964 0 -1 64 7658 0 1 0 4051 35 0 0 25 0 10 0 59314538 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4127984624 0 4 0 25189583 0 0 0 -1 0 0 0
[pid=21531/tid=21533] ppid=21526 vsize=851072 CPUtime=4.75
/proc/21531/task/21533/stat : 21533 (java) S 21526 21526 19964 0 -1 64 54075 0 0 0 443 32 0 0 16 0 10 0 59314539 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21534] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21534/stat : 21534 (java) S 21526 21526 19964 0 -1 64 13 0 0 0 0 0 0 0 16 0 10 0 59314540 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21535] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21535/stat : 21535 (java) S 21526 21526 19964 0 -1 64 8 0 0 0 0 0 0 0 16 0 10 0 59314540 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21536] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21536/stat : 21536 (java) S 21526 21526 19964 0 -1 64 0 0 0 0 0 0 0 0 18 0 10 0 59314542 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21537] ppid=21526 vsize=851072 CPUtime=4.62
/proc/21531/task/21537/stat : 21537 (java) S 21526 21526 19964 0 -1 64 3535 0 0 0 449 13 0 0 16 0 10 0 59314546 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21538] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21538/stat : 21538 (java) S 21526 21526 19964 0 -1 64 1 0 0 0 0 0 0 0 19 0 10 0 59314546 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21539] ppid=21526 vsize=851072 CPUtime=0.08
/proc/21531/task/21539/stat : 21539 (java) S 21526 21526 19964 0 -1 64 1 0 0 0 2 6 0 0 15 0 10 0 59314546 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21543] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21543/stat : 21543 (java) S 21526 21526 19964 0 -1 64 0 0 0 0 0 0 0 0 17 0 10 0 59319272 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 50.35
Current children cumulated vsize (KiB) 856428

Solver just ended. Dumping a history of the last processes samples

[startup+51.2021 s]
/proc/loadavg: 2.04 2.01 2.00 3/83 21574
/proc/meminfo: memFree=1480968/2055920 swapFree=4192812/4192956
[pid=21526] ppid=21524 vsize=5356 CPUtime=0
/proc/21526/stat : 21526 (concrete.sh) S 21524 21526 19964 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59314536 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 217871606596 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21526/statm: 1339 244 201 169 0 50 0
[pid=21531] ppid=21526 vsize=851072 CPUtime=50.46
/proc/21531/stat : 21531 (java) S 21526 21526 19964 0 -1 0 65974 0 1 0 4958 88 0 0 18 0 10 0 59314537 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 18446744073709551615 0 0 17 1 0 0
/proc/21531/statm: 212768 64556 2634 9 0 207246 0
[pid=21531/tid=21532] ppid=21526 vsize=851072 CPUtime=40.97
/proc/21531/task/21532/stat : 21532 (java) R 21526 21526 19964 0 -1 64 7658 0 1 0 4062 35 0 0 25 0 10 0 59314538 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4128141960 0 4 0 25189583 0 0 0 -1 0 0 0
[pid=21531/tid=21533] ppid=21526 vsize=851072 CPUtime=4.75
/proc/21531/task/21533/stat : 21533 (java) S 21526 21526 19964 0 -1 64 54075 0 0 0 443 32 0 0 16 0 10 0 59314539 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21534] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21534/stat : 21534 (java) S 21526 21526 19964 0 -1 64 13 0 0 0 0 0 0 0 16 0 10 0 59314540 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21535] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21535/stat : 21535 (java) S 21526 21526 19964 0 -1 64 8 0 0 0 0 0 0 0 16 0 10 0 59314540 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21536] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21536/stat : 21536 (java) S 21526 21526 19964 0 -1 64 0 0 0 0 0 0 0 0 18 0 10 0 59314542 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21537] ppid=21526 vsize=851072 CPUtime=4.62
/proc/21531/task/21537/stat : 21537 (java) S 21526 21526 19964 0 -1 64 3535 0 0 0 449 13 0 0 16 0 10 0 59314546 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21538] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21538/stat : 21538 (java) S 21526 21526 19964 0 -1 64 1 0 0 0 0 0 0 0 19 0 10 0 59314546 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21539] ppid=21526 vsize=851072 CPUtime=0.08
/proc/21531/task/21539/stat : 21539 (java) S 21526 21526 19964 0 -1 64 1 0 0 0 2 6 0 0 15 0 10 0 59314546 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21543] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21543/stat : 21543 (java) S 21526 21526 19964 0 -1 64 0 0 0 0 0 0 0 0 17 0 10 0 59319272 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 50.46
Current children cumulated vsize (KiB) 856428

[startup+52.8025 s]
/proc/loadavg: 2.04 2.01 2.00 3/83 21574
/proc/meminfo: memFree=1480456/2055920 swapFree=4192812/4192956
[pid=21526] ppid=21524 vsize=5356 CPUtime=0
/proc/21526/stat : 21526 (concrete.sh) S 21524 21526 19964 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59314536 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 217871606596 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21526/statm: 1339 244 201 169 0 50 0
[pid=21531] ppid=21526 vsize=851072 CPUtime=51.35
/proc/21531/stat : 21531 (java) S 21526 21526 19964 0 -1 0 65974 0 1 0 5047 88 0 0 18 0 10 0 59314537 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 18446744073709551615 0 0 17 1 0 0
/proc/21531/statm: 212768 64556 2634 9 0 207246 0
[pid=21531/tid=21532] ppid=21526 vsize=851072 CPUtime=41.87
/proc/21531/task/21532/stat : 21532 (java) R 21526 21526 19964 0 -1 64 7658 0 1 0 4152 35 0 0 25 0 10 0 59314538 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4128141981 0 4 0 25189583 0 0 0 -1 0 0 0
[pid=21531/tid=21533] ppid=21526 vsize=851072 CPUtime=4.75
/proc/21531/task/21533/stat : 21533 (java) S 21526 21526 19964 0 -1 64 54075 0 0 0 443 32 0 0 16 0 10 0 59314539 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21534] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21534/stat : 21534 (java) S 21526 21526 19964 0 -1 64 13 0 0 0 0 0 0 0 16 0 10 0 59314540 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21535] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21535/stat : 21535 (java) S 21526 21526 19964 0 -1 64 8 0 0 0 0 0 0 0 16 0 10 0 59314540 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21536] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21536/stat : 21536 (java) S 21526 21526 19964 0 -1 64 0 0 0 0 0 0 0 0 18 0 10 0 59314542 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21537] ppid=21526 vsize=851072 CPUtime=4.62
/proc/21531/task/21537/stat : 21537 (java) S 21526 21526 19964 0 -1 64 3535 0 0 0 449 13 0 0 16 0 10 0 59314546 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21538] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21538/stat : 21538 (java) S 21526 21526 19964 0 -1 64 1 0 0 0 0 0 0 0 19 0 10 0 59314546 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21539] ppid=21526 vsize=851072 CPUtime=0.08
/proc/21531/task/21539/stat : 21539 (java) S 21526 21526 19964 0 -1 64 1 0 0 0 2 6 0 0 15 0 10 0 59314546 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21543] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21543/stat : 21543 (java) S 21526 21526 19964 0 -1 64 0 0 0 0 0 0 0 0 17 0 10 0 59319272 871497728 64556 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 51.35
Current children cumulated vsize (KiB) 856428

[startup+54.4027 s]
/proc/loadavg: 2.04 2.01 2.00 3/82 21574
/proc/meminfo: memFree=1480136/2055920 swapFree=4192812/4192956
[pid=21526] ppid=21524 vsize=5356 CPUtime=0
/proc/21526/stat : 21526 (concrete.sh) S 21524 21526 19964 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59314536 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 217871606596 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21526/statm: 1339 244 201 169 0 50 0
[pid=21531] ppid=21526 vsize=851072 CPUtime=52.92
/proc/21531/stat : 21531 (java) S 21526 21526 19964 0 -1 0 66012 0 1 0 5204 88 0 0 18 0 9 0 59314537 871497728 64588 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 18446744073709551615 0 0 17 1 0 0
/proc/21531/statm: 212768 64588 2634 9 0 207246 0
[pid=21531/tid=21532] ppid=21526 vsize=851072 CPUtime=43.3
/proc/21531/task/21532/stat : 21532 (java) R 21526 21526 19964 0 -1 64 7663 0 1 0 4295 35 0 0 25 0 9 0 59314538 871497728 64588 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 0 0 0
[pid=21531/tid=21533] ppid=21526 vsize=851072 CPUtime=4.87
/proc/21531/task/21533/stat : 21533 (java) R 21526 21526 19964 0 -1 64 54107 0 0 0 455 32 0 0 16 0 9 0 59314539 871497728 64588 996147200 134512640 134550932 4294955408 18446744073709551615 102549400 0 0 0 25189583 0 0 0 -1 0 0 0
[pid=21531/tid=21534] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21534/stat : 21534 (java) S 21526 21526 19964 0 -1 64 13 0 0 0 0 0 0 0 16 0 9 0 59314540 871497728 64588 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 0 0 0
[pid=21531/tid=21535] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21535/stat : 21535 (java) S 21526 21526 19964 0 -1 64 8 0 0 0 0 0 0 0 16 0 9 0 59314540 871497728 64588 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 0 0 0
[pid=21531/tid=21536] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21536/stat : 21536 (java) S 21526 21526 19964 0 -1 64 0 0 0 0 0 0 0 0 18 0 9 0 59314542 871497728 64588 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21537] ppid=21526 vsize=851072 CPUtime=4.63
/proc/21531/task/21537/stat : 21537 (java) S 21526 21526 19964 0 -1 64 3535 0 0 0 450 13 0 0 15 0 9 0 59314546 871497728 64588 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 0 0 0
[pid=21531/tid=21538] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21538/stat : 21538 (java) S 21526 21526 19964 0 -1 64 1 0 0 0 0 0 0 0 19 0 9 0 59314546 871497728 64588 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21539] ppid=21526 vsize=851072 CPUtime=0.08
/proc/21531/task/21539/stat : 21539 (java) S 21526 21526 19964 0 -1 64 1 0 0 0 2 6 0 0 15 0 9 0 59314546 871497728 64588 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 52.92
Current children cumulated vsize (KiB) 856428

[startup+54.8027 s]
/proc/loadavg: 2.04 2.01 2.00 3/82 21574
/proc/meminfo: memFree=1480136/2055920 swapFree=4192812/4192956
[pid=21526] ppid=21524 vsize=5356 CPUtime=0
/proc/21526/stat : 21526 (concrete.sh) S 21524 21526 19964 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59314536 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 217871606596 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21526/statm: 1339 244 201 169 0 50 0
[pid=21531] ppid=21526 vsize=851072 CPUtime=53.31
/proc/21531/stat : 21531 (java) S 21526 21526 19964 0 -1 0 66014 0 1 0 5243 88 0 0 18 0 9 0 59314537 871497728 64588 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 18446744073709551615 0 0 17 1 0 0
/proc/21531/statm: 212768 64588 2634 9 0 207246 0
[pid=21531/tid=21532] ppid=21526 vsize=851072 CPUtime=43.64
/proc/21531/task/21532/stat : 21532 (java) R 21526 21526 19964 0 -1 64 7665 0 1 0 4329 35 0 0 25 0 9 0 59314538 871497728 64588 996147200 134512640 134550932 4294955408 18446744073709551615 102094016 0 4 0 25189583 0 0 0 -1 0 0 0
[pid=21531/tid=21533] ppid=21526 vsize=851072 CPUtime=4.92
/proc/21531/task/21533/stat : 21533 (java) S 21526 21526 19964 0 -1 64 54107 0 0 0 460 32 0 0 16 0 9 0 59314539 871497728 64588 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 0 0 0 -1 0 0 0
[pid=21531/tid=21534] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21534/stat : 21534 (java) S 21526 21526 19964 0 -1 64 13 0 0 0 0 0 0 0 16 0 9 0 59314540 871497728 64588 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 0 0 0
[pid=21531/tid=21535] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21535/stat : 21535 (java) S 21526 21526 19964 0 -1 64 8 0 0 0 0 0 0 0 16 0 9 0 59314540 871497728 64588 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 0 0 0
[pid=21531/tid=21536] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21536/stat : 21536 (java) S 21526 21526 19964 0 -1 64 0 0 0 0 0 0 0 0 18 0 9 0 59314542 871497728 64588 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21537] ppid=21526 vsize=851072 CPUtime=4.63
/proc/21531/task/21537/stat : 21537 (java) S 21526 21526 19964 0 -1 64 3535 0 0 0 450 13 0 0 15 0 9 0 59314546 871497728 64588 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 0 0 0
[pid=21531/tid=21538] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21538/stat : 21538 (java) S 21526 21526 19964 0 -1 64 1 0 0 0 0 0 0 0 19 0 9 0 59314546 871497728 64588 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21539] ppid=21526 vsize=851072 CPUtime=0.08
/proc/21531/task/21539/stat : 21539 (java) S 21526 21526 19964 0 -1 64 1 0 0 0 2 6 0 0 15 0 9 0 59314546 871497728 64588 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 53.31
Current children cumulated vsize (KiB) 856428

[startup+55.0027 s]
/proc/loadavg: 2.04 2.01 2.00 2/82 21575
/proc/meminfo: memFree=1479944/2055920 swapFree=4192812/4192956
[pid=21526] ppid=21524 vsize=5356 CPUtime=0
/proc/21526/stat : 21526 (concrete.sh) S 21524 21526 19964 0 -1 4194304 341 291 0 0 0 0 0 0 18 0 1 0 59314536 5484544 244 996147200 4194304 4889804 548682068816 18446744073709551615 217871606596 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21526/statm: 1339 244 201 169 0 50 0
[pid=21531] ppid=21526 vsize=851072 CPUtime=53.51
/proc/21531/stat : 21531 (java) S 21526 21526 19964 0 -1 0 66020 0 1 0 5263 88 0 0 18 0 9 0 59314537 871497728 64590 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 18446744073709551615 0 0 17 1 0 0
/proc/21531/statm: 212768 64590 2634 9 0 207246 0
[pid=21531/tid=21532] ppid=21526 vsize=851072 CPUtime=43.8
/proc/21531/task/21532/stat : 21532 (java) S 21526 21526 19964 0 -1 64 7671 0 1 0 4345 35 0 0 25 0 9 0 59314538 871497728 64590 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 0 0 0
[pid=21531/tid=21533] ppid=21526 vsize=851072 CPUtime=4.96
/proc/21531/task/21533/stat : 21533 (java) S 21526 21526 19964 0 -1 64 54107 0 0 0 464 32 0 0 16 0 9 0 59314539 871497728 64590 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 0 0 25189583 0 0 0 -1 0 0 0
[pid=21531/tid=21534] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21534/stat : 21534 (java) S 21526 21526 19964 0 -1 64 13 0 0 0 0 0 0 0 16 0 9 0 59314540 871497728 64590 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 0 0 0
[pid=21531/tid=21535] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21535/stat : 21535 (java) S 21526 21526 19964 0 -1 64 8 0 0 0 0 0 0 0 16 0 9 0 59314540 871497728 64590 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 0 0 0
[pid=21531/tid=21536] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21536/stat : 21536 (java) S 21526 21526 19964 0 -1 64 0 0 0 0 0 0 0 0 18 0 9 0 59314542 871497728 64590 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21537] ppid=21526 vsize=851072 CPUtime=4.63
/proc/21531/task/21537/stat : 21537 (java) S 21526 21526 19964 0 -1 64 3535 0 0 0 450 13 0 0 15 0 9 0 59314546 871497728 64590 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 0 0 0
[pid=21531/tid=21538] ppid=21526 vsize=851072 CPUtime=0
/proc/21531/task/21538/stat : 21538 (java) S 21526 21526 19964 0 -1 64 1 0 0 0 0 0 0 0 19 0 9 0 59314546 871497728 64590 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
[pid=21531/tid=21539] ppid=21526 vsize=851072 CPUtime=0.08
/proc/21531/task/21539/stat : 21539 (java) S 21526 21526 19964 0 -1 64 1 0 0 0 2 6 0 0 15 0 9 0 59314546 871497728 64590 996147200 134512640 134550932 4294955408 18446744073709551615 4294960144 0 4 0 25189583 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 53.51
Current children cumulated vsize (KiB) 856428

Child status: 0
Real time (s): 55.0958
CPU time (s): 53.5779
CPU user time (s): 52.643
CPU system time (s): 0.934857
CPU usage (%): 97.2449
Max. virtual memory (cumulated for all children) (KiB): 856428

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 52.643
system time used= 0.934857
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 66665
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 21805
involuntary context switches= 21631

runsolver used 0.093985 second user time and 0.203968 second system time

The end

Launcher Data 

Begin job on node28 at 2009-07-16 07:34:39
IDJOB=2083112
IDBENCH=57705
IDSOLVER=735
FILE ID=node28/2083112-1247722479
PBS_JOBID= 9521462
Free space on /tmp= 66008 MiB

SOLVER NAME= Concrete 2009-06-10
BENCH NAME= CPAI08/csp/fapp/fapp08/normalized-fapp08-0700-8.xml
COMMAND LINE= HOME/concrete.sh -c BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-2083112-1247722479/watcher-2083112-1247722479 -o /tmp/evaluation-result-2083112-1247722479/solver-2083112-1247722479 -C 1800 -W 2000 -M 900  HOME/concrete.sh -c HOME/instance-2083112-1247722479.xml

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 900 MiB
MAX NB THREAD= 0

MD5SUM BENCH= 76f5c8323181a5b7c43a501cf4eaddc1
RANDOM SEED=167783226

node28.alineos.net Linux 2.6.9-22.EL.rootsmp #1 SMP Mon Oct 3 08:59:52 CEST 2005

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.230
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5914.62
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.230
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5586.94
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:      2055920 kB
MemFree:       1630832 kB
Buffers:         46292 kB
Cached:         167500 kB
SwapCached:          0 kB
Active:         223868 kB
Inactive:       137916 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      2055920 kB
LowFree:       1630832 kB
SwapTotal:     4192956 kB
SwapFree:      4192812 kB
Dirty:            2248 kB
Writeback:           0 kB
Mapped:         168404 kB
Slab:            48068 kB
Committed_AS:   789572 kB
PageTables:       2148 kB
VmallocTotal: 536870911 kB
VmallocUsed:    264960 kB
VmallocChunk: 536605679 kB
HugePages_Total:     0
HugePages_Free:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 66004 MiB
End job on node28 at 2009-07-16 07:35:34