Trace number 211163

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds, and are wall clock time (not CPU time).

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
CSP4J - MAC 2006-12-19? (TO) 1800.07 1802.2

General information on the benchmark

NamepseudoSeries/uclid/
dlx1c-rwmem1-ucl.xml
MD5SUMcad824bee8074e1420d713f6438fb5bd
Bench CategoryN-ARY-INT (n-ary constraints in intension)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark12.84
SatisfiableNO
(Un)Satisfiability was provedNO
Number of variables7578
Number of constraints21400
Maximum constraint arity11
Maximum domain size2
Number of constraints which are defined in extension0
Number of constraints which are defined in intension21400
Global constraints used (with number of constraints)

Solver Data (download as text)



Verifier Data (download as text)




ERROR: Unexpected answer ! (SAT/UNSAT expected)
Got answer: 



Watcher Data (download as text)




runsolver version 3.1.3 (c) roussel@cril.univ-artois.fr

command line: runsolver --timestamp -w ROOT/results/node56/watcher-211163-1168205841 -o ROOT/results/node56/solver-211163-1168205841 -C 1800 -M 900 /tmp/evaluation/211163-1168205841/cspfj.sh -competition /tmp/evaluation/211163-1168205841/unknown.xml 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 921600 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 972800 KiB
Current StackSize limit: 10240 KiB

/proc/loadavg: 1.37 1.83 1.89 2/65 15032
/proc/meminfo: memFree=1492072/2055920 swapFree=4184772/4192956
[pid=15031] ppid=15029 vsize=5352 CPUtime=0
/proc/15031/stat : 15031 (cspfj.sh) R 15029 15031 14409 0 -1 4194304 291 128 0 0 0 0 0 0 18 0 1 0 168982274 5480448 236 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 253306792435 0 0 4100 65536 0 0 0 17 1 0 0
/proc/15031/statm: 1338 238 199 169 0 49 0

[startup+0.102682 s]
/proc/loadavg: 1.37 1.83 1.89 2/65 15032
/proc/meminfo: memFree=1492072/2055920 swapFree=4184772/4192956
[pid=15031] ppid=15029 vsize=5352 CPUtime=0
/proc/15031/stat : 15031 (cspfj.sh) S 15029 15031 14409 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 168982274 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/15031/statm: 1338 240 201 169 0 49 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 5352

[startup+0.510718 s]
/proc/loadavg: 1.37 1.83 1.89 2/65 15032
/proc/meminfo: memFree=1492072/2055920 swapFree=4184772/4192956
[pid=15031] ppid=15029 vsize=5352 CPUtime=0
/proc/15031/stat : 15031 (cspfj.sh) S 15029 15031 14409 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 168982274 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/15031/statm: 1338 240 201 169 0 49 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 5352

[startup+1.3328 s]
/proc/loadavg: 1.37 1.83 1.89 2/73 15041
/proc/meminfo: memFree=1479392/2055920 swapFree=4184772/4192956
[pid=15031] ppid=15029 vsize=5352 CPUtime=0
/proc/15031/stat : 15031 (cspfj.sh) S 15029 15031 14409 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 168982274 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/15031/statm: 1338 240 201 169 0 49 0
[pid=15034] ppid=15031 vsize=865780 CPUtime=1.26
/proc/15034/stat : 15034 (java) R 15031 15031 14409 0 -1 0 5377 0 1 0 121 5 0 0 17 0 8 0 168982279 886558720 5068 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4154015731 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/15034/statm: 216445 5068 1828 14 0 203707 0
[pid=15034/tid=15035] ppid=15031 vsize=865780 CPUtime=0.15
/proc/15034/task/15035/stat : 15035 (java) S 15031 15031 14409 0 -1 64 1243 0 0 0 15 0 0 0 16 0 8 0 168982282 886558720 5068 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15036] ppid=15031 vsize=865780 CPUtime=0
/proc/15034/task/15036/stat : 15036 (java) S 15031 15031 14409 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 168982283 886558720 5068 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15037] ppid=15031 vsize=865780 CPUtime=0
/proc/15034/task/15037/stat : 15037 (java) S 15031 15031 14409 0 -1 64 7 0 0 0 0 0 0 0 15 0 8 0 168982283 886558720 5068 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15038] ppid=15031 vsize=865780 CPUtime=0
/proc/15034/task/15038/stat : 15038 (java) S 15031 15031 14409 0 -1 64 0 0 0 0 0 0 0 0 23 0 8 0 168982286 886558720 5068 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15039] ppid=15031 vsize=865780 CPUtime=0.24
/proc/15034/task/15039/stat : 15039 (java) S 15031 15031 14409 0 -1 64 542 0 0 0 23 1 0 0 16 0 8 0 168982286 886558720 5068 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15040] ppid=15031 vsize=865780 CPUtime=0
/proc/15034/task/15040/stat : 15040 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 24 0 8 0 168982286 886558720 5068 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15041] ppid=15031 vsize=865780 CPUtime=0
/proc/15034/task/15041/stat : 15041 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 168982286 886558720 5068 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 1.26
Current children cumulated vsize (KiB) 871132

[startup+2.98615 s]
/proc/loadavg: 1.37 1.83 1.89 2/73 15041
/proc/meminfo: memFree=1475232/2055920 swapFree=4184772/4192956
[pid=15031] ppid=15029 vsize=5352 CPUtime=0
/proc/15031/stat : 15031 (cspfj.sh) S 15029 15031 14409 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 168982274 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/15031/statm: 1338 240 201 169 0 49 0
[pid=15034] ppid=15031 vsize=866860 CPUtime=2.93
/proc/15034/stat : 15034 (java) R 15031 15031 14409 0 -1 0 6903 0 1 0 282 11 0 0 17 0 8 0 168982279 887664640 6510 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4152819582 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/15034/statm: 216715 6510 1828 14 0 203977 0
[pid=15034/tid=15035] ppid=15031 vsize=866860 CPUtime=0.32
/proc/15034/task/15035/stat : 15035 (java) S 15031 15031 14409 0 -1 64 2095 0 0 0 31 1 0 0 16 0 8 0 168982282 887664640 6510 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15036] ppid=15031 vsize=866860 CPUtime=0
/proc/15034/task/15036/stat : 15036 (java) S 15031 15031 14409 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 168982283 887664640 6510 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15037] ppid=15031 vsize=866860 CPUtime=0
/proc/15034/task/15037/stat : 15037 (java) S 15031 15031 14409 0 -1 64 7 0 0 0 0 0 0 0 15 0 8 0 168982283 887664640 6510 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15038] ppid=15031 vsize=866860 CPUtime=0
/proc/15034/task/15038/stat : 15038 (java) S 15031 15031 14409 0 -1 64 0 0 0 0 0 0 0 0 23 0 8 0 168982286 887664640 6510 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15039] ppid=15031 vsize=866860 CPUtime=0.32
/proc/15034/task/15039/stat : 15039 (java) S 15031 15031 14409 0 -1 64 582 0 0 0 31 1 0 0 16 0 8 0 168982286 887664640 6510 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15040] ppid=15031 vsize=866860 CPUtime=0
/proc/15034/task/15040/stat : 15040 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 24 0 8 0 168982286 887664640 6510 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15041] ppid=15031 vsize=866860 CPUtime=0
/proc/15034/task/15041/stat : 15041 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 168982286 887664640 6510 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 2.93
Current children cumulated vsize (KiB) 872212

[startup+6.27047 s]
/proc/loadavg: 1.34 1.81 1.89 2/73 15041
/proc/meminfo: memFree=1465504/2055920 swapFree=4184772/4192956
[pid=15031] ppid=15029 vsize=5352 CPUtime=0
/proc/15031/stat : 15031 (cspfj.sh) S 15029 15031 14409 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 168982274 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/15031/statm: 1338 240 201 169 0 49 0
[pid=15034] ppid=15031 vsize=866860 CPUtime=6.21
/proc/15034/stat : 15034 (java) S 15031 15031 14409 0 -1 0 9245 0 1 0 602 19 0 0 16 0 8 0 168982279 887664640 8791 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/15034/statm: 216715 8791 1828 14 0 203977 0
[pid=15034/tid=15035] ppid=15031 vsize=866860 CPUtime=0.56
/proc/15034/task/15035/stat : 15035 (java) R 15031 15031 14409 0 -1 64 3478 0 0 0 54 2 0 0 16 0 8 0 168982282 887664640 8791 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4155528745 0 0 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15036] ppid=15031 vsize=866860 CPUtime=0
/proc/15034/task/15036/stat : 15036 (java) S 15031 15031 14409 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 168982283 887664640 8791 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15037] ppid=15031 vsize=866860 CPUtime=0
/proc/15034/task/15037/stat : 15037 (java) S 15031 15031 14409 0 -1 64 7 0 0 0 0 0 0 0 15 0 8 0 168982283 887664640 8791 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15038] ppid=15031 vsize=866860 CPUtime=0
/proc/15034/task/15038/stat : 15038 (java) S 15031 15031 14409 0 -1 64 0 0 0 0 0 0 0 0 23 0 8 0 168982286 887664640 8791 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15039] ppid=15031 vsize=866860 CPUtime=0.32
/proc/15034/task/15039/stat : 15039 (java) S 15031 15031 14409 0 -1 64 583 0 0 0 31 1 0 0 16 0 8 0 168982286 887664640 8791 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15040] ppid=15031 vsize=866860 CPUtime=0
/proc/15034/task/15040/stat : 15040 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 24 0 8 0 168982286 887664640 8791 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15041] ppid=15031 vsize=866860 CPUtime=0
/proc/15034/task/15041/stat : 15041 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 168982286 887664640 8791 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 6.21
Current children cumulated vsize (KiB) 872212

[startup+12.7559 s]
/proc/loadavg: 1.31 1.80 1.88 2/73 15041
/proc/meminfo: memFree=1453928/2055920 swapFree=4184772/4192956
[pid=15031] ppid=15029 vsize=5352 CPUtime=0
/proc/15031/stat : 15031 (cspfj.sh) S 15029 15031 14409 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 168982274 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/15031/statm: 1338 240 201 169 0 49 0
[pid=15034] ppid=15031 vsize=866860 CPUtime=12.67
/proc/15034/stat : 15034 (java) R 15031 15031 14409 0 -1 0 11853 0 1 0 1240 27 0 0 17 0 8 0 168982279 887664640 11350 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4073816970 0 4 0 23759 18446744073709551615 0 0 17 1 0 0
/proc/15034/statm: 216715 11350 1828 14 0 203977 0
[pid=15034/tid=15035] ppid=15031 vsize=866860 CPUtime=0.73
/proc/15034/task/15035/stat : 15035 (java) S 15031 15031 14409 0 -1 64 5096 0 0 0 71 2 0 0 16 0 8 0 168982282 887664640 11350 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15036] ppid=15031 vsize=866860 CPUtime=0
/proc/15034/task/15036/stat : 15036 (java) S 15031 15031 14409 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 168982283 887664640 11350 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15037] ppid=15031 vsize=866860 CPUtime=0
/proc/15034/task/15037/stat : 15037 (java) S 15031 15031 14409 0 -1 64 7 0 0 0 0 0 0 0 15 0 8 0 168982283 887664640 11350 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15038] ppid=15031 vsize=866860 CPUtime=0
/proc/15034/task/15038/stat : 15038 (java) S 15031 15031 14409 0 -1 64 0 0 0 0 0 0 0 0 23 0 8 0 168982286 887664640 11350 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15039] ppid=15031 vsize=866860 CPUtime=0.32
/proc/15034/task/15039/stat : 15039 (java) S 15031 15031 14409 0 -1 64 583 0 0 0 31 1 0 0 16 0 8 0 168982286 887664640 11350 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15040] ppid=15031 vsize=866860 CPUtime=0
/proc/15034/task/15040/stat : 15040 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 24 0 8 0 168982286 887664640 11350 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15041] ppid=15031 vsize=866860 CPUtime=0
/proc/15034/task/15041/stat : 15041 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 168982286 887664640 11350 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 12.67
Current children cumulated vsize (KiB) 872212

[startup+25.6072 s]
/proc/loadavg: 1.24 1.76 1.87 2/73 15041
/proc/meminfo: memFree=1440744/2055920 swapFree=4184772/4192956
[pid=15031] ppid=15029 vsize=5352 CPUtime=0
/proc/15031/stat : 15031 (cspfj.sh) S 15029 15031 14409 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 168982274 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/15031/statm: 1338 240 201 169 0 49 0
[pid=15034] ppid=15031 vsize=867884 CPUtime=25.49
/proc/15034/stat : 15034 (java) R 15031 15031 14409 0 -1 0 15312 0 1 0 2507 42 0 0 17 0 8 0 168982279 888713216 14767 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4074173283 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/15034/statm: 216971 14767 1828 14 0 204233 0
[pid=15034/tid=15035] ppid=15031 vsize=867884 CPUtime=1.14
/proc/15034/task/15035/stat : 15035 (java) S 15031 15031 14409 0 -1 64 7187 0 0 0 111 3 0 0 16 0 8 0 168982282 888713216 14767 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15036] ppid=15031 vsize=867884 CPUtime=0
/proc/15034/task/15036/stat : 15036 (java) S 15031 15031 14409 0 -1 64 11 0 0 0 0 0 0 0 17 0 8 0 168982283 888713216 14767 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15037] ppid=15031 vsize=867884 CPUtime=0
/proc/15034/task/15037/stat : 15037 (java) S 15031 15031 14409 0 -1 64 7 0 0 0 0 0 0 0 15 0 8 0 168982283 888713216 14767 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15038] ppid=15031 vsize=867884 CPUtime=0
/proc/15034/task/15038/stat : 15038 (java) S 15031 15031 14409 0 -1 64 0 0 0 0 0 0 0 0 23 0 8 0 168982286 888713216 14767 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15039] ppid=15031 vsize=867884 CPUtime=0.32
/proc/15034/task/15039/stat : 15039 (java) S 15031 15031 14409 0 -1 64 583 0 0 0 31 1 0 0 16 0 8 0 168982286 888713216 14767 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15040] ppid=15031 vsize=867884 CPUtime=0
/proc/15034/task/15040/stat : 15040 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 24 0 8 0 168982286 888713216 14767 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15041] ppid=15031 vsize=867884 CPUtime=0
/proc/15034/task/15041/stat : 15041 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 168982286 888713216 14767 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 25.49
Current children cumulated vsize (KiB) 873236

[startup+51.2237 s]
/proc/loadavg: 1.16 1.70 1.84 2/73 15041
/proc/meminfo: memFree=1420904/2055920 swapFree=4184772/4192956
[pid=15031] ppid=15029 vsize=5352 CPUtime=0
/proc/15031/stat : 15031 (cspfj.sh) S 15029 15031 14409 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 168982274 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/15031/statm: 1338 240 201 169 0 49 0
[pid=15034] ppid=15031 vsize=868204 CPUtime=51.03
/proc/15034/stat : 15034 (java) R 15031 15031 14409 0 -1 0 20561 0 1 0 5043 60 0 0 18 0 8 0 168982279 889040896 19564 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4073330699 0 4 0 23759 18446744073709551615 0 0 17 1 0 0
/proc/15034/statm: 217051 19564 1828 14 0 204313 0
[pid=15034/tid=15035] ppid=15031 vsize=868204 CPUtime=2.06
/proc/15034/task/15035/stat : 15035 (java) S 15031 15031 14409 0 -1 64 10197 0 0 0 202 4 0 0 16 0 8 0 168982282 889040896 19564 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15036] ppid=15031 vsize=868204 CPUtime=0
/proc/15034/task/15036/stat : 15036 (java) S 15031 15031 14409 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 168982283 889040896 19564 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15037] ppid=15031 vsize=868204 CPUtime=0
/proc/15034/task/15037/stat : 15037 (java) S 15031 15031 14409 0 -1 64 7 0 0 0 0 0 0 0 15 0 8 0 168982283 889040896 19564 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15038] ppid=15031 vsize=868204 CPUtime=0
/proc/15034/task/15038/stat : 15038 (java) S 15031 15031 14409 0 -1 64 0 0 0 0 0 0 0 0 23 0 8 0 168982286 889040896 19564 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15039] ppid=15031 vsize=868204 CPUtime=0.32
/proc/15034/task/15039/stat : 15039 (java) S 15031 15031 14409 0 -1 64 583 0 0 0 31 1 0 0 16 0 8 0 168982286 889040896 19564 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15040] ppid=15031 vsize=868204 CPUtime=0
/proc/15034/task/15040/stat : 15040 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 24 0 8 0 168982286 889040896 19564 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15041] ppid=15031 vsize=868204 CPUtime=0
/proc/15034/task/15041/stat : 15041 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 168982286 889040896 19564 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 51.03
Current children cumulated vsize (KiB) 873556

[startup+102.449 s]
/proc/loadavg: 1.07 1.59 1.80 2/73 15041
/proc/meminfo: memFree=1393704/2055920 swapFree=4184772/4192956
[pid=15031] ppid=15029 vsize=5352 CPUtime=0
/proc/15031/stat : 15031 (cspfj.sh) S 15029 15031 14409 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 168982274 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/15031/statm: 1338 240 201 169 0 49 0
[pid=15034] ppid=15031 vsize=869868 CPUtime=102.14
/proc/15034/stat : 15034 (java) R 15031 15031 14409 0 -1 0 36949 0 1 0 10118 96 0 0 17 0 8 0 168982279 890744832 26437 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4073837040 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/15034/statm: 217467 26437 1828 14 0 204729 0
[pid=15034/tid=15035] ppid=15031 vsize=869868 CPUtime=3.37

################
# More data... #
################

/proc/loadavg: 1.00 1.00 1.14 2/73 15239
/proc/meminfo: memFree=1368688/2055920 swapFree=4184772/4192956
[pid=15031] ppid=15029 vsize=5352 CPUtime=0
/proc/15031/stat : 15031 (cspfj.sh) S 15029 15031 14409 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 168982274 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/15031/statm: 1338 240 201 169 0 49 0
[pid=15034] ppid=15031 vsize=871988 CPUtime=1541.85
/proc/15034/stat : 15034 (java) R 15031 15031 14409 0 -1 0 49725 0 1 0 154035 150 0 0 25 0 8 0 168982279 892915712 32431 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4152040759 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/15034/statm: 217997 32431 1840 14 0 205259 0
[pid=15034/tid=15035] ppid=15031 vsize=871988 CPUtime=12.9
/proc/15034/task/15035/stat : 15035 (java) S 15031 15031 14409 0 -1 64 18725 0 0 0 1279 11 0 0 16 0 8 0 168982282 892915712 32431 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15036] ppid=15031 vsize=871988 CPUtime=0.01
/proc/15034/task/15036/stat : 15036 (java) S 15031 15031 14409 0 -1 64 11 0 0 0 1 0 0 0 16 0 8 0 168982283 892915712 32431 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15037] ppid=15031 vsize=871988 CPUtime=0
/proc/15034/task/15037/stat : 15037 (java) S 15031 15031 14409 0 -1 64 7 0 0 0 0 0 0 0 16 0 8 0 168982283 892915712 32431 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15038] ppid=15031 vsize=871988 CPUtime=0
/proc/15034/task/15038/stat : 15038 (java) S 15031 15031 14409 0 -1 64 0 0 0 0 0 0 0 0 23 0 8 0 168982286 892915712 32431 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15039] ppid=15031 vsize=871988 CPUtime=3.19
/proc/15034/task/15039/stat : 15039 (java) S 15031 15031 14409 0 -1 64 2541 0 0 0 303 16 0 0 15 0 8 0 168982286 892915712 32431 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15040] ppid=15031 vsize=871988 CPUtime=0
/proc/15034/task/15040/stat : 15040 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 24 0 8 0 168982286 892915712 32431 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15041] ppid=15031 vsize=871988 CPUtime=0
/proc/15034/task/15041/stat : 15041 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 168982286 892915712 32431 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 1541.85
Current children cumulated vsize (KiB) 877340

[startup+1603.78 s]
/proc/loadavg: 1.00 1.00 1.13 2/73 15239
/proc/meminfo: memFree=1368112/2055920 swapFree=4184772/4192956
[pid=15031] ppid=15029 vsize=5352 CPUtime=0
/proc/15031/stat : 15031 (cspfj.sh) S 15029 15031 14409 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 168982274 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/15031/statm: 1338 240 201 169 0 49 0
[pid=15034] ppid=15031 vsize=872276 CPUtime=1601.82
/proc/15034/stat : 15034 (java) R 15031 15031 14409 0 -1 0 49987 0 1 0 160030 152 0 0 25 0 8 0 168982279 893210624 32625 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4074213806 0 4 0 23759 18446744073709551615 0 0 17 1 0 0
/proc/15034/statm: 218069 32625 1840 14 0 205331 0
[pid=15034/tid=15035] ppid=15031 vsize=872276 CPUtime=14.23
/proc/15034/task/15035/stat : 15035 (java) S 15031 15031 14409 0 -1 64 18725 0 0 0 1411 12 0 0 16 0 8 0 168982282 893210624 32625 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15036] ppid=15031 vsize=872276 CPUtime=0.01
/proc/15034/task/15036/stat : 15036 (java) S 15031 15031 14409 0 -1 64 11 0 0 0 1 0 0 0 16 0 8 0 168982283 893210624 32625 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15037] ppid=15031 vsize=872276 CPUtime=0
/proc/15034/task/15037/stat : 15037 (java) S 15031 15031 14409 0 -1 64 7 0 0 0 0 0 0 0 16 0 8 0 168982283 893210624 32625 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15038] ppid=15031 vsize=872276 CPUtime=0
/proc/15034/task/15038/stat : 15038 (java) S 15031 15031 14409 0 -1 64 0 0 0 0 0 0 0 0 23 0 8 0 168982286 893210624 32625 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15039] ppid=15031 vsize=872276 CPUtime=3.34
/proc/15034/task/15039/stat : 15039 (java) S 15031 15031 14409 0 -1 64 2734 0 0 0 318 16 0 0 15 0 8 0 168982286 893210624 32625 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15040] ppid=15031 vsize=872276 CPUtime=0
/proc/15034/task/15040/stat : 15040 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 24 0 8 0 168982286 893210624 32625 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15041] ppid=15031 vsize=872276 CPUtime=0
/proc/15034/task/15041/stat : 15041 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 168982286 893210624 32625 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 1601.82
Current children cumulated vsize (KiB) 877628

[startup+1663.81 s]
/proc/loadavg: 1.00 1.00 1.12 2/73 15239
/proc/meminfo: memFree=1367664/2055920 swapFree=4184772/4192956
[pid=15031] ppid=15029 vsize=5352 CPUtime=0
/proc/15031/stat : 15031 (cspfj.sh) S 15029 15031 14409 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 168982274 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/15031/statm: 1338 240 201 169 0 49 0
[pid=15034] ppid=15031 vsize=872424 CPUtime=1661.8
/proc/15034/stat : 15034 (java) R 15031 15031 14409 0 -1 0 50105 0 1 0 166027 153 0 0 19 0 8 0 168982279 893362176 32738 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4074221853 0 4 0 23759 18446744073709551615 0 0 17 1 0 0
/proc/15034/statm: 218106 32738 1840 14 0 205368 0
[pid=15034/tid=15035] ppid=15031 vsize=872424 CPUtime=14.34
/proc/15034/task/15035/stat : 15035 (java) S 15031 15031 14409 0 -1 64 18725 0 0 0 1422 12 0 0 16 0 8 0 168982282 893362176 32738 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15036] ppid=15031 vsize=872424 CPUtime=0.01
/proc/15034/task/15036/stat : 15036 (java) S 15031 15031 14409 0 -1 64 11 0 0 0 1 0 0 0 16 0 8 0 168982283 893362176 32738 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15037] ppid=15031 vsize=872424 CPUtime=0
/proc/15034/task/15037/stat : 15037 (java) S 15031 15031 14409 0 -1 64 7 0 0 0 0 0 0 0 16 0 8 0 168982283 893362176 32738 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15038] ppid=15031 vsize=872424 CPUtime=0
/proc/15034/task/15038/stat : 15038 (java) S 15031 15031 14409 0 -1 64 0 0 0 0 0 0 0 0 23 0 8 0 168982286 893362176 32738 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15039] ppid=15031 vsize=872424 CPUtime=3.51
/proc/15034/task/15039/stat : 15039 (java) S 15031 15031 14409 0 -1 64 2847 0 0 0 334 17 0 0 15 0 8 0 168982286 893362176 32738 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15040] ppid=15031 vsize=872424 CPUtime=0
/proc/15034/task/15040/stat : 15040 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 24 0 8 0 168982286 893362176 32738 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15041] ppid=15031 vsize=872424 CPUtime=0
/proc/15034/task/15041/stat : 15041 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 168982286 893362176 32738 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 1661.8
Current children cumulated vsize (KiB) 877776

[startup+1723.84 s]
/proc/loadavg: 1.00 1.00 1.11 2/73 15239
/proc/meminfo: memFree=1367088/2055920 swapFree=4184772/4192956
[pid=15031] ppid=15029 vsize=5352 CPUtime=0
/proc/15031/stat : 15031 (cspfj.sh) S 15029 15031 14409 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 168982274 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/15031/statm: 1338 240 201 169 0 49 0
[pid=15034] ppid=15031 vsize=872556 CPUtime=1721.78
/proc/15034/stat : 15034 (java) R 15031 15031 14409 0 -1 0 50245 0 1 0 172023 155 0 0 25 0 8 0 168982279 893497344 32872 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4074213806 0 4 0 23759 18446744073709551615 0 0 17 1 0 0
/proc/15034/statm: 218139 32872 1840 14 0 205401 0
[pid=15034/tid=15035] ppid=15031 vsize=872556 CPUtime=14.47
/proc/15034/task/15035/stat : 15035 (java) S 15031 15031 14409 0 -1 64 18725 0 0 0 1435 12 0 0 16 0 8 0 168982282 893497344 32872 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15036] ppid=15031 vsize=872556 CPUtime=0.01
/proc/15034/task/15036/stat : 15036 (java) S 15031 15031 14409 0 -1 64 11 0 0 0 1 0 0 0 16 0 8 0 168982283 893497344 32872 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15037] ppid=15031 vsize=872556 CPUtime=0
/proc/15034/task/15037/stat : 15037 (java) S 15031 15031 14409 0 -1 64 7 0 0 0 0 0 0 0 16 0 8 0 168982283 893497344 32872 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15038] ppid=15031 vsize=872556 CPUtime=0
/proc/15034/task/15038/stat : 15038 (java) S 15031 15031 14409 0 -1 64 0 0 0 0 0 0 0 0 23 0 8 0 168982286 893497344 32872 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15039] ppid=15031 vsize=872556 CPUtime=3.71
/proc/15034/task/15039/stat : 15039 (java) S 15031 15031 14409 0 -1 64 2981 0 0 0 353 18 0 0 15 0 8 0 168982286 893497344 32872 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15040] ppid=15031 vsize=872556 CPUtime=0
/proc/15034/task/15040/stat : 15040 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 24 0 8 0 168982286 893497344 32872 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15041] ppid=15031 vsize=872556 CPUtime=0
/proc/15034/task/15041/stat : 15041 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 168982286 893497344 32872 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 1721.78
Current children cumulated vsize (KiB) 877908

[startup+1783.88 s]
/proc/loadavg: 1.00 1.00 1.10 2/73 15239
/proc/meminfo: memFree=1366704/2055920 swapFree=4184772/4192956
[pid=15031] ppid=15029 vsize=5352 CPUtime=0
/proc/15031/stat : 15031 (cspfj.sh) S 15029 15031 14409 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 168982274 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/15031/statm: 1338 240 201 169 0 49 0
[pid=15034] ppid=15031 vsize=872688 CPUtime=1781.77
/proc/15034/stat : 15034 (java) R 15031 15031 14409 0 -1 0 50342 0 1 0 178021 156 0 0 25 0 8 0 168982279 893632512 32962 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4074271561 0 4 0 23759 18446744073709551615 0 0 17 1 0 0
/proc/15034/statm: 218172 32962 1840 14 0 205434 0
[pid=15034/tid=15035] ppid=15031 vsize=872688 CPUtime=14.63
/proc/15034/task/15035/stat : 15035 (java) S 15031 15031 14409 0 -1 64 18725 0 0 0 1451 12 0 0 16 0 8 0 168982282 893632512 32962 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15036] ppid=15031 vsize=872688 CPUtime=0.01
/proc/15034/task/15036/stat : 15036 (java) S 15031 15031 14409 0 -1 64 11 0 0 0 1 0 0 0 16 0 8 0 168982283 893632512 32962 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15037] ppid=15031 vsize=872688 CPUtime=0
/proc/15034/task/15037/stat : 15037 (java) S 15031 15031 14409 0 -1 64 7 0 0 0 0 0 0 0 16 0 8 0 168982283 893632512 32962 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15038] ppid=15031 vsize=872688 CPUtime=0
/proc/15034/task/15038/stat : 15038 (java) S 15031 15031 14409 0 -1 64 0 0 0 0 0 0 0 0 23 0 8 0 168982286 893632512 32962 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15039] ppid=15031 vsize=872688 CPUtime=3.85
/proc/15034/task/15039/stat : 15039 (java) S 15031 15031 14409 0 -1 64 3071 0 0 0 366 19 0 0 15 0 8 0 168982286 893632512 32962 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15040] ppid=15031 vsize=872688 CPUtime=0
/proc/15034/task/15040/stat : 15040 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 24 0 8 0 168982286 893632512 32962 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15041] ppid=15031 vsize=872688 CPUtime=0
/proc/15034/task/15041/stat : 15041 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 168982286 893632512 32962 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 1781.77
Current children cumulated vsize (KiB) 878040



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1802.18 s]
/proc/loadavg: 1.00 1.00 1.09 2/73 15239
/proc/meminfo: memFree=1366576/2055920 swapFree=4184772/4192956
[pid=15031] ppid=15029 vsize=5352 CPUtime=0
/proc/15031/stat : 15031 (cspfj.sh) S 15029 15031 14409 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 168982274 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 253307183940 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/15031/statm: 1338 240 201 169 0 49 0
[pid=15034] ppid=15031 vsize=872688 CPUtime=1800.07
/proc/15034/stat : 15034 (java) R 15031 15031 14409 0 -1 0 50362 0 1 0 179851 156 0 0 23 0 8 0 168982279 893632512 32980 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4074221847 0 4 0 23759 18446744073709551615 0 0 17 1 0 0
/proc/15034/statm: 218172 32980 1840 14 0 205434 0
[pid=15034/tid=15035] ppid=15031 vsize=872688 CPUtime=14.67
/proc/15034/task/15035/stat : 15035 (java) S 15031 15031 14409 0 -1 64 18725 0 0 0 1455 12 0 0 16 0 8 0 168982282 893632512 32980 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15036] ppid=15031 vsize=872688 CPUtime=0.01
/proc/15034/task/15036/stat : 15036 (java) S 15031 15031 14409 0 -1 64 11 0 0 0 1 0 0 0 16 0 8 0 168982283 893632512 32980 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15037] ppid=15031 vsize=872688 CPUtime=0
/proc/15034/task/15037/stat : 15037 (java) S 15031 15031 14409 0 -1 64 7 0 0 0 0 0 0 0 16 0 8 0 168982283 893632512 32980 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15038] ppid=15031 vsize=872688 CPUtime=0
/proc/15034/task/15038/stat : 15038 (java) S 15031 15031 14409 0 -1 64 0 0 0 0 0 0 0 0 23 0 8 0 168982286 893632512 32980 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15039] ppid=15031 vsize=872688 CPUtime=3.87
/proc/15034/task/15039/stat : 15039 (java) S 15031 15031 14409 0 -1 64 3089 0 0 0 368 19 0 0 15 0 8 0 168982286 893632512 32980 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=15034/tid=15040] ppid=15031 vsize=872688 CPUtime=0
/proc/15034/task/15040/stat : 15040 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 24 0 8 0 168982286 893632512 32980 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=15034/tid=15041] ppid=15031 vsize=872688 CPUtime=0
/proc/15034/task/15041/stat : 15041 (java) S 15031 15031 14409 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 168982286 893632512 32980 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 1800.07
Current children cumulated vsize (KiB) 878040

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Child ended because it received signal 15 (SIGTERM)

!!! problem with CPU time !!!
wait4(...,&childrusage) returns 15031 and gives childrusage.ru_utime.tv_sec=0 childrusage.ru_utime.tv_usec=1999 childrusage.ru_stime.tv_sec=0 childrusage.ru_stime.tv_usec=1999
CPU time returned by wait4() is 0.003998
while last known CPU time is 1800.07

Solver probably didn't/couldn't wait for its children
Using last known CPU time as value...

Real time (s): 1802.2
CPU time (s): 1800.07
CPU user time (s): 1798.51
CPU system time (s): 1.56
CPU usage (%): 99.8817
Max. virtual memory (cumulated for all children) (KiB): 878248

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.001999
system time used= 0.001999
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 443
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 18
involuntary context switches= 4

runsolver used 1.9787 s user time and 4.48532 s system time

The end



Launcher Data (download as text)




Begin job on node56 on Sun Jan  7 21:37:21 UTC 2007


IDJOB= 211163
IDBENCH= 4673
FILE ID= node56/211163-1168205841

PBS_JOBID= 3480767

Free space on /tmp= 66554 MiB

BENCH NAME= HOME/pub/bench/CPAI06/pseudoSeries/uclid/dlx1c-rwmem1-ucl.xml
COMMAND LINE= /tmp/evaluation/211163-1168205841/cspfj.sh -competition /tmp/evaluation/211163-1168205841/unknown.xml
RUNSOLVER COMMAND LINE= runsolver  --timestamp  -w ROOT/results/node56/watcher-211163-1168205841 -o ROOT/results/node56/solver-211163-1168205841 -C 1800 -M 900  /tmp/evaluation/211163-1168205841/cspfj.sh -competition /tmp/evaluation/211163-1168205841/unknown.xml

META MD5SUM SOLVER= ae5901f6cce23734dab709fa25e92adb
MD5SUM BENCH=  cad824bee8074e1420d713f6438fb5bd

RANDOM SEED= 153857442

TIME LIMIT= 1800 seconds

MEMORY LIMIT= 900 MiB


/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.232
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5914.62
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.232
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5586.94
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:      2055920 kB
MemFree:       1492552 kB
Buffers:         47380 kB
Cached:         436580 kB
SwapCached:       2104 kB
Active:         125424 kB
Inactive:       368312 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      2055920 kB
LowFree:       1492552 kB
SwapTotal:     4192956 kB
SwapFree:      4184772 kB
Dirty:            3608 kB
Writeback:           0 kB
Mapped:          16628 kB
Slab:            55600 kB
Committed_AS:  3280584 kB
PageTables:       1472 kB
VmallocTotal: 536870911 kB
VmallocUsed:    264952 kB
VmallocChunk: 536605679 kB
HugePages_Total:     0
HugePages_Free:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 66554 MiB



End job on node56 on Sun Jan  7 22:07:24 UTC 2007