Trace number 270891

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds, and are wall clock time (not CPU time).

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
CSP4J - MAC 2007-01-16? (TO) 1800.02 1816.76

General information on the benchmark

NamepseudoSeries/uclid/
ooo-burch-dill-4-accl.xml
MD5SUM86c8ce99ba43ad8e61bc7c9612a8229b
Bench CategoryN-ARY-INT (n-ary constraints in intension)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark12.7421
SatisfiableNO
(Un)Satisfiability was provedNO
Number of variables3939
Number of constraints10411
Maximum constraint arity13
Maximum domain size2
Number of constraints which are defined in extension0
Number of constraints which are defined in intension10411
Global constraints used (with number of constraints)

Solver Data (download as text)



Verifier Data (download as text)




ERROR: Unexpected answer ! (SAT/UNSAT expected)
Got answer: 



Watcher Data (download as text)




runsolver version 3.1.3 (c) roussel@cril.univ-artois.fr

command line: runsolver --timestamp -w ROOT/results/node19/watcher-270891-1169151296 -o ROOT/results/node19/solver-270891-1169151296 -C 1800 -M 900 /tmp/evaluation/270891-1169151296/cspfj.sh -competition /tmp/evaluation/270891-1169151296/unknown.xml 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 921600 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 972800 KiB
Current StackSize limit: 10240 KiB

/proc/loadavg: 0.85 0.98 1.17 4/82 21572
/proc/meminfo: memFree=1564728/2055920 swapFree=4171264/4192956
[pid=21571] ppid=21569 vsize=5352 CPUtime=0
/proc/21571/stat : 21571 (cspfj.sh) S 21569 21571 21452 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 263524298 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 255236563780 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21571/statm: 1338 240 201 169 0 49 0
[pid=21574] ppid=21571 vsize=168 CPUtime=0
/proc/21574/stat : 21574 (java) R 21571 21571 21452 0 -1 0 39 0 0 0 0 0 0 0 19 0 1 0 263524299 172032 23 18446744073709551615 134512640 134570532 4294956672 18446744073709551615 7484196 0 0 4096 0 0 0 0 17 1 0 0
/proc/21574/statm: 54 30 25 14 0 2 0

[startup+0.10304 s]
/proc/loadavg: 0.85 0.98 1.17 4/82 21572
/proc/meminfo: memFree=1564728/2055920 swapFree=4171264/4192956
[pid=21571] ppid=21569 vsize=5352 CPUtime=0
/proc/21571/stat : 21571 (cspfj.sh) S 21569 21571 21452 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 263524298 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 255236563780 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21571/statm: 1338 240 201 169 0 49 0
[pid=21574] ppid=21571 vsize=860404 CPUtime=0.04
/proc/21574/stat : 21574 (java) R 21571 21571 21452 0 -1 0 1959 0 1 0 3 1 0 0 20 0 4 0 263524299 881053696 1705 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4153611072 0 4 0 7368 18446744073709551615 0 0 17 0 0 0
/proc/21574/statm: 215101 1705 1167 14 0 202684 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 865756

[startup+0.512115 s]
/proc/loadavg: 0.85 0.98 1.17 4/82 21572
/proc/meminfo: memFree=1564728/2055920 swapFree=4171264/4192956
[pid=21571] ppid=21569 vsize=5352 CPUtime=0
/proc/21571/stat : 21571 (cspfj.sh) S 21569 21571 21452 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 263524298 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 255236563780 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21571/statm: 1338 240 201 169 0 49 0
[pid=21574] ppid=21571 vsize=865020 CPUtime=0.34
/proc/21574/stat : 21574 (java) S 21571 21571 21452 0 -1 0 3648 0 1 0 30 4 0 0 16 0 8 0 263524299 885780480 3389 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/21574/statm: 216255 3389 1741 14 0 203525 0
Current children cumulated CPU time (s) 0.34
Current children cumulated vsize (KiB) 870372

[startup+1.33327 s]
/proc/loadavg: 0.85 0.98 1.17 5/102 21633
/proc/meminfo: memFree=1519832/2055920 swapFree=4171264/4192956
[pid=21571] ppid=21569 vsize=5352 CPUtime=0
/proc/21571/stat : 21571 (cspfj.sh) S 21569 21571 21452 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 263524298 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 255236563780 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21571/statm: 1338 240 201 169 0 49 0
[pid=21574] ppid=21571 vsize=866172 CPUtime=1.16
/proc/21574/stat : 21574 (java) R 21571 21571 21452 0 -1 0 5093 0 1 0 108 8 0 0 20 0 8 0 263524299 886960128 4798 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 7917109 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/21574/statm: 216543 4798 1829 14 0 203804 0
[pid=21574/tid=21575] ppid=21571 vsize=866172 CPUtime=0.1
/proc/21574/task/21575/stat : 21575 (java) S 21571 21571 21452 0 -1 64 859 0 0 0 10 0 0 0 16 0 8 0 263524306 886960128 4798 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21576] ppid=21571 vsize=866172 CPUtime=0
/proc/21574/task/21576/stat : 21576 (java) S 21571 21571 21452 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 263524307 886960128 4798 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21577] ppid=21571 vsize=866172 CPUtime=0
/proc/21574/task/21577/stat : 21577 (java) S 21571 21571 21452 0 -1 64 7 0 0 0 0 0 0 0 16 0 8 0 263524307 886960128 4798 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21578] ppid=21571 vsize=866172 CPUtime=0
/proc/21574/task/21578/stat : 21578 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 19 0 8 0 263524314 886960128 4798 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21579] ppid=21571 vsize=866172 CPUtime=0.24
/proc/21574/task/21579/stat : 21579 (java) S 21571 21571 21452 0 -1 64 694 0 0 0 22 2 0 0 15 0 8 0 263524314 886960128 4798 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21580] ppid=21571 vsize=866172 CPUtime=0
/proc/21574/task/21580/stat : 21580 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 20 0 8 0 263524314 886960128 4798 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21581] ppid=21571 vsize=866172 CPUtime=0
/proc/21574/task/21581/stat : 21581 (java) S 21571 21571 21452 0 -1 64 0 0 0 0 0 0 0 0 15 0 8 0 263524314 886960128 4798 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 1.16
Current children cumulated vsize (KiB) 871524

[startup+2.98058 s]
/proc/loadavg: 0.85 0.98 1.17 3/102 21633
/proc/meminfo: memFree=1496984/2055920 swapFree=4171264/4192956
[pid=21571] ppid=21569 vsize=5352 CPUtime=0
/proc/21571/stat : 21571 (cspfj.sh) S 21569 21571 21452 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 263524298 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 255236563780 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21571/statm: 1338 240 201 169 0 49 0
[pid=21574] ppid=21571 vsize=866480 CPUtime=2.8
/proc/21574/stat : 21574 (java) R 21571 21571 21452 0 -1 0 6724 0 1 0 266 14 0 0 25 0 8 0 263524299 887275520 6347 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4152166574 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/21574/statm: 216620 6347 1829 14 0 203881 0
[pid=21574/tid=21575] ppid=21571 vsize=866480 CPUtime=0.25
/proc/21574/task/21575/stat : 21575 (java) S 21571 21571 21452 0 -1 64 1834 0 0 0 24 1 0 0 16 0 8 0 263524306 887275520 6347 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21576] ppid=21571 vsize=866480 CPUtime=0
/proc/21574/task/21576/stat : 21576 (java) S 21571 21571 21452 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 263524307 887275520 6347 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21577] ppid=21571 vsize=866480 CPUtime=0
/proc/21574/task/21577/stat : 21577 (java) S 21571 21571 21452 0 -1 64 7 0 0 0 0 0 0 0 16 0 8 0 263524307 887275520 6347 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21578] ppid=21571 vsize=866480 CPUtime=0
/proc/21574/task/21578/stat : 21578 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 19 0 8 0 263524314 887275520 6347 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21579] ppid=21571 vsize=866480 CPUtime=0.35
/proc/21574/task/21579/stat : 21579 (java) S 21571 21571 21452 0 -1 64 745 0 0 0 32 3 0 0 15 0 8 0 263524314 887275520 6347 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21580] ppid=21571 vsize=866480 CPUtime=0
/proc/21574/task/21580/stat : 21580 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 20 0 8 0 263524314 887275520 6347 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21581] ppid=21571 vsize=866480 CPUtime=0
/proc/21574/task/21581/stat : 21581 (java) S 21571 21571 21452 0 -1 64 0 0 0 0 0 0 0 0 15 0 8 0 263524314 887275520 6347 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 2.8
Current children cumulated vsize (KiB) 871832

[startup+6.18219 s]
/proc/loadavg: 0.94 1.00 1.17 3/102 21633
/proc/meminfo: memFree=1476520/2055920 swapFree=4171264/4192956
[pid=21571] ppid=21569 vsize=5352 CPUtime=0
/proc/21571/stat : 21571 (cspfj.sh) S 21569 21571 21452 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 263524298 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 255236563780 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21571/statm: 1338 240 201 169 0 49 0
[pid=21574] ppid=21571 vsize=866640 CPUtime=5.97
/proc/21574/stat : 21574 (java) R 21571 21571 21452 0 -1 0 9207 0 1 0 574 23 0 0 25 0 8 0 263524299 887439360 8763 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4073838593 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/21574/statm: 216660 8763 1829 14 0 203921 0
[pid=21574/tid=21575] ppid=21571 vsize=866640 CPUtime=0.59
/proc/21574/task/21575/stat : 21575 (java) S 21571 21571 21452 0 -1 64 3326 0 0 0 57 2 0 0 16 0 8 0 263524306 887439360 8763 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21576] ppid=21571 vsize=866640 CPUtime=0
/proc/21574/task/21576/stat : 21576 (java) S 21571 21571 21452 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 263524307 887439360 8763 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21577] ppid=21571 vsize=866640 CPUtime=0
/proc/21574/task/21577/stat : 21577 (java) S 21571 21571 21452 0 -1 64 7 0 0 0 0 0 0 0 16 0 8 0 263524307 887439360 8763 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21578] ppid=21571 vsize=866640 CPUtime=0
/proc/21574/task/21578/stat : 21578 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 19 0 8 0 263524314 887439360 8763 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21579] ppid=21571 vsize=866640 CPUtime=0.35
/proc/21574/task/21579/stat : 21579 (java) S 21571 21571 21452 0 -1 64 746 0 0 0 32 3 0 0 15 0 8 0 263524314 887439360 8763 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21580] ppid=21571 vsize=866640 CPUtime=0
/proc/21574/task/21580/stat : 21580 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 20 0 8 0 263524314 887439360 8763 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21581] ppid=21571 vsize=866640 CPUtime=0
/proc/21574/task/21581/stat : 21581 (java) S 21571 21571 21452 0 -1 64 0 0 0 0 0 0 0 0 15 0 8 0 263524314 887439360 8763 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 5.97
Current children cumulated vsize (KiB) 871992

[startup+12.6724 s]
/proc/loadavg: 1.10 1.03 1.18 3/102 21633
/proc/meminfo: memFree=1452008/2055920 swapFree=4171264/4192956
[pid=21571] ppid=21569 vsize=5352 CPUtime=0
/proc/21571/stat : 21571 (cspfj.sh) S 21569 21571 21452 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 263524298 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 255236563780 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21571/statm: 1338 240 201 169 0 49 0
[pid=21574] ppid=21571 vsize=866640 CPUtime=12.43
/proc/21574/stat : 21574 (java) R 21571 21571 21452 0 -1 0 11969 0 1 0 1209 34 0 0 25 0 8 0 263524299 887439360 11472 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4073838616 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/21574/statm: 216660 11472 1829 14 0 203921 0
[pid=21574/tid=21575] ppid=21571 vsize=866640 CPUtime=0.78
/proc/21574/task/21575/stat : 21575 (java) S 21571 21571 21452 0 -1 64 5136 0 0 0 76 2 0 0 16 0 8 0 263524306 887439360 11472 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21576] ppid=21571 vsize=866640 CPUtime=0
/proc/21574/task/21576/stat : 21576 (java) S 21571 21571 21452 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 263524307 887439360 11472 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21577] ppid=21571 vsize=866640 CPUtime=0
/proc/21574/task/21577/stat : 21577 (java) S 21571 21571 21452 0 -1 64 7 0 0 0 0 0 0 0 16 0 8 0 263524307 887439360 11472 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21578] ppid=21571 vsize=866640 CPUtime=0
/proc/21574/task/21578/stat : 21578 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 19 0 8 0 263524314 887439360 11472 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21579] ppid=21571 vsize=866640 CPUtime=0.35
/proc/21574/task/21579/stat : 21579 (java) S 21571 21571 21452 0 -1 64 746 0 0 0 32 3 0 0 15 0 8 0 263524314 887439360 11472 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21580] ppid=21571 vsize=866640 CPUtime=0
/proc/21574/task/21580/stat : 21580 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 20 0 8 0 263524314 887439360 11472 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21581] ppid=21571 vsize=866640 CPUtime=0
/proc/21574/task/21581/stat : 21581 (java) S 21571 21571 21452 0 -1 64 0 0 0 0 0 0 0 0 15 0 8 0 263524314 887439360 11472 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 12.43
Current children cumulated vsize (KiB) 871992

[startup+25.5618 s]
/proc/loadavg: 1.30 1.08 1.19 3/102 21633
/proc/meminfo: memFree=1422376/2055920 swapFree=4171264/4192956
[pid=21571] ppid=21569 vsize=5352 CPUtime=0
/proc/21571/stat : 21571 (cspfj.sh) S 21569 21571 21452 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 263524298 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 255236563780 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21571/statm: 1338 240 201 169 0 49 0
[pid=21574] ppid=21571 vsize=866932 CPUtime=25.24
/proc/21574/stat : 21574 (java) R 21571 21571 21452 0 -1 0 15983 0 1 0 2475 49 0 0 25 0 8 0 263524299 887738368 15440 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4073813561 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/21574/statm: 216733 15440 1835 14 0 203994 0
[pid=21574/tid=21575] ppid=21571 vsize=866932 CPUtime=1.46
/proc/21574/task/21575/stat : 21575 (java) S 21571 21571 21452 0 -1 64 7697 0 0 0 143 3 0 0 16 0 8 0 263524306 887738368 15440 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21576] ppid=21571 vsize=866932 CPUtime=0
/proc/21574/task/21576/stat : 21576 (java) S 21571 21571 21452 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 263524307 887738368 15440 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21577] ppid=21571 vsize=866932 CPUtime=0
/proc/21574/task/21577/stat : 21577 (java) S 21571 21571 21452 0 -1 64 7 0 0 0 0 0 0 0 16 0 8 0 263524307 887738368 15440 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21578] ppid=21571 vsize=866932 CPUtime=0
/proc/21574/task/21578/stat : 21578 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 19 0 8 0 263524314 887738368 15440 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21579] ppid=21571 vsize=866932 CPUtime=0.35
/proc/21574/task/21579/stat : 21579 (java) S 21571 21571 21452 0 -1 64 746 0 0 0 32 3 0 0 16 0 8 0 263524314 887738368 15440 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21580] ppid=21571 vsize=866932 CPUtime=0
/proc/21574/task/21580/stat : 21580 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 20 0 8 0 263524314 887738368 15440 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21581] ppid=21571 vsize=866932 CPUtime=0
/proc/21574/task/21581/stat : 21581 (java) S 21571 21571 21452 0 -1 64 0 0 0 0 0 0 0 0 15 0 8 0 263524314 887738368 15440 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 25.24
Current children cumulated vsize (KiB) 872284

[startup+51.2287 s]
/proc/loadavg: 1.60 1.16 1.22 3/102 21633
/proc/meminfo: memFree=1389032/2055920 swapFree=4171264/4192956
[pid=21571] ppid=21569 vsize=5352 CPUtime=0
/proc/21571/stat : 21571 (cspfj.sh) S 21569 21571 21452 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 263524298 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 255236563780 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21571/statm: 1338 240 201 169 0 49 0
[pid=21574] ppid=21571 vsize=867624 CPUtime=50.74
/proc/21574/stat : 21574 (java) R 21571 21571 21452 0 -1 0 18906 0 1 0 5016 58 0 0 25 0 8 0 263524299 888446976 17946 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4072919695 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/21574/statm: 216906 17946 1841 14 0 204167 0
[pid=21574/tid=21575] ppid=21571 vsize=867624 CPUtime=2.18
/proc/21574/task/21575/stat : 21575 (java) S 21571 21571 21452 0 -1 64 9734 0 0 0 214 4 0 0 16 0 8 0 263524306 888446976 17946 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21576] ppid=21571 vsize=867624 CPUtime=0
/proc/21574/task/21576/stat : 21576 (java) S 21571 21571 21452 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 263524307 888446976 17946 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21577] ppid=21571 vsize=867624 CPUtime=0
/proc/21574/task/21577/stat : 21577 (java) S 21571 21571 21452 0 -1 64 7 0 0 0 0 0 0 0 16 0 8 0 263524307 888446976 17946 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21578] ppid=21571 vsize=867624 CPUtime=0
/proc/21574/task/21578/stat : 21578 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 19 0 8 0 263524314 888446976 17946 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21579] ppid=21571 vsize=867624 CPUtime=0.9
/proc/21574/task/21579/stat : 21579 (java) S 21571 21571 21452 0 -1 64 1321 0 0 0 85 5 0 0 17 0 8 0 263524314 888446976 17946 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21580] ppid=21571 vsize=867624 CPUtime=0
/proc/21574/task/21580/stat : 21580 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 20 0 8 0 263524314 888446976 17946 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21581] ppid=21571 vsize=867624 CPUtime=0
/proc/21574/task/21581/stat : 21581 (java) S 21571 21571 21452 0 -1 64 0 0 0 0 0 0 0 0 15 0 8 0 263524314 888446976 17946 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 50.74
Current children cumulated vsize (KiB) 872976

[startup+102.517 s]

################
# More data... #
################

/proc/loadavg: 2.02 2.03 1.88 3/102 21707
/proc/meminfo: memFree=798304/2055920 swapFree=4171264/4192956
[pid=21571] ppid=21569 vsize=5352 CPUtime=0
/proc/21571/stat : 21571 (cspfj.sh) S 21569 21571 21452 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 263524298 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 255236563780 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21571/statm: 1338 240 201 169 0 49 0
[pid=21574] ppid=21571 vsize=868820 CPUtime=1529.62
/proc/21574/stat : 21574 (java) R 21571 21571 21452 0 -1 0 20903 0 1 0 152880 82 0 0 25 0 8 0 263524299 889671680 19023 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4074280538 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/21574/statm: 217205 19023 1841 14 0 204466 0
[pid=21574/tid=21575] ppid=21571 vsize=868820 CPUtime=10.8
/proc/21574/task/21575/stat : 21575 (java) S 21571 21571 21452 0 -1 64 9734 0 0 0 1069 11 0 0 16 0 8 0 263524306 889671680 19023 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21576] ppid=21571 vsize=868820 CPUtime=0
/proc/21574/task/21576/stat : 21576 (java) S 21571 21571 21452 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 263524307 889671680 19023 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21577] ppid=21571 vsize=868820 CPUtime=0
/proc/21574/task/21577/stat : 21577 (java) S 21571 21571 21452 0 -1 64 7 0 0 0 0 0 0 0 16 0 8 0 263524307 889671680 19023 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21578] ppid=21571 vsize=868820 CPUtime=0
/proc/21574/task/21578/stat : 21578 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 19 0 8 0 263524314 889671680 19023 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21579] ppid=21571 vsize=868820 CPUtime=2.13
/proc/21574/task/21579/stat : 21579 (java) S 21571 21571 21452 0 -1 64 2398 0 0 0 202 11 0 0 16 0 8 0 263524314 889671680 19023 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21580] ppid=21571 vsize=868820 CPUtime=0
/proc/21574/task/21580/stat : 21580 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 20 0 8 0 263524314 889671680 19023 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21581] ppid=21571 vsize=868820 CPUtime=0
/proc/21574/task/21581/stat : 21581 (java) S 21571 21571 21452 0 -1 64 0 0 0 0 0 0 0 0 15 0 8 0 263524314 889671680 19023 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 1529.62
Current children cumulated vsize (KiB) 874172

[startup+1603.56 s]
/proc/loadavg: 2.01 2.02 1.89 3/102 21707
/proc/meminfo: memFree=797856/2055920 swapFree=4171264/4192956
[pid=21571] ppid=21569 vsize=5352 CPUtime=0
/proc/21571/stat : 21571 (cspfj.sh) S 21569 21571 21452 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 263524298 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 255236563780 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21571/statm: 1338 240 201 169 0 49 0
[pid=21574] ppid=21571 vsize=868952 CPUtime=1589.01
/proc/21574/stat : 21574 (java) R 21571 21571 21452 0 -1 0 20970 0 1 0 158818 83 0 0 25 0 8 0 263524299 889806848 19057 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4074242288 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/21574/statm: 217238 19057 1841 14 0 204499 0
[pid=21574/tid=21575] ppid=21571 vsize=868952 CPUtime=11.1
/proc/21574/task/21575/stat : 21575 (java) S 21571 21571 21452 0 -1 64 9734 0 0 0 1099 11 0 0 16 0 8 0 263524306 889806848 19057 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21576] ppid=21571 vsize=868952 CPUtime=0
/proc/21574/task/21576/stat : 21576 (java) S 21571 21571 21452 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 263524307 889806848 19057 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21577] ppid=21571 vsize=868952 CPUtime=0
/proc/21574/task/21577/stat : 21577 (java) S 21571 21571 21452 0 -1 64 7 0 0 0 0 0 0 0 16 0 8 0 263524307 889806848 19057 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21578] ppid=21571 vsize=868952 CPUtime=0
/proc/21574/task/21578/stat : 21578 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 19 0 8 0 263524314 889806848 19057 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21579] ppid=21571 vsize=868952 CPUtime=2.19
/proc/21574/task/21579/stat : 21579 (java) S 21571 21571 21452 0 -1 64 2432 0 0 0 207 12 0 0 16 0 8 0 263524314 889806848 19057 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21580] ppid=21571 vsize=868952 CPUtime=0
/proc/21574/task/21580/stat : 21580 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 20 0 8 0 263524314 889806848 19057 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21581] ppid=21571 vsize=868952 CPUtime=0
/proc/21574/task/21581/stat : 21581 (java) S 21571 21571 21452 0 -1 64 0 0 0 0 0 0 0 0 15 0 8 0 263524314 889806848 19057 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 1589.01
Current children cumulated vsize (KiB) 874304

[startup+1663.6 s]
/proc/loadavg: 2.00 2.02 1.89 3/102 21707
/proc/meminfo: memFree=797344/2055920 swapFree=4171264/4192956
[pid=21571] ppid=21569 vsize=5352 CPUtime=0
/proc/21571/stat : 21571 (cspfj.sh) S 21569 21571 21452 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 263524298 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 255236563780 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21571/statm: 1338 240 201 169 0 49 0
[pid=21574] ppid=21571 vsize=868952 CPUtime=1648.43
/proc/21574/stat : 21574 (java) R 21571 21571 21452 0 -1 0 20997 0 1 0 164759 84 0 0 25 0 8 0 263524299 889806848 19064 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4075337678 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/21574/statm: 217238 19064 1841 14 0 204499 0
[pid=21574/tid=21575] ppid=21571 vsize=868952 CPUtime=11.29
/proc/21574/task/21575/stat : 21575 (java) S 21571 21571 21452 0 -1 64 9734 0 0 0 1118 11 0 0 16 0 8 0 263524306 889806848 19064 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21576] ppid=21571 vsize=868952 CPUtime=0
/proc/21574/task/21576/stat : 21576 (java) S 21571 21571 21452 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 263524307 889806848 19064 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21577] ppid=21571 vsize=868952 CPUtime=0
/proc/21574/task/21577/stat : 21577 (java) S 21571 21571 21452 0 -1 64 7 0 0 0 0 0 0 0 16 0 8 0 263524307 889806848 19064 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21578] ppid=21571 vsize=868952 CPUtime=0
/proc/21574/task/21578/stat : 21578 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 19 0 8 0 263524314 889806848 19064 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21579] ppid=21571 vsize=868952 CPUtime=2.2
/proc/21574/task/21579/stat : 21579 (java) S 21571 21571 21452 0 -1 64 2439 0 0 0 208 12 0 0 16 0 8 0 263524314 889806848 19064 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21580] ppid=21571 vsize=868952 CPUtime=0
/proc/21574/task/21580/stat : 21580 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 20 0 8 0 263524314 889806848 19064 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21581] ppid=21571 vsize=868952 CPUtime=0
/proc/21574/task/21581/stat : 21581 (java) S 21571 21571 21452 0 -1 64 0 0 0 0 0 0 0 0 15 0 8 0 263524314 889806848 19064 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 1648.43
Current children cumulated vsize (KiB) 874304

[startup+1723.63 s]
/proc/loadavg: 2.04 2.03 1.90 3/102 21707
/proc/meminfo: memFree=796832/2055920 swapFree=4171264/4192956
[pid=21571] ppid=21569 vsize=5352 CPUtime=0
/proc/21571/stat : 21571 (cspfj.sh) S 21569 21571 21452 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 263524298 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 255236563780 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21571/statm: 1338 240 201 169 0 49 0
[pid=21574] ppid=21571 vsize=868952 CPUtime=1707.86
/proc/21574/stat : 21574 (java) R 21571 21571 21452 0 -1 0 21044 0 1 0 170702 84 0 0 25 0 8 0 263524299 889806848 19097 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4074280503 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/21574/statm: 217238 19097 1841 14 0 204499 0
[pid=21574/tid=21575] ppid=21571 vsize=868952 CPUtime=11.42
/proc/21574/task/21575/stat : 21575 (java) S 21571 21571 21452 0 -1 64 9734 0 0 0 1130 12 0 0 16 0 8 0 263524306 889806848 19097 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21576] ppid=21571 vsize=868952 CPUtime=0
/proc/21574/task/21576/stat : 21576 (java) S 21571 21571 21452 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 263524307 889806848 19097 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21577] ppid=21571 vsize=868952 CPUtime=0
/proc/21574/task/21577/stat : 21577 (java) S 21571 21571 21452 0 -1 64 7 0 0 0 0 0 0 0 16 0 8 0 263524307 889806848 19097 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21578] ppid=21571 vsize=868952 CPUtime=0
/proc/21574/task/21578/stat : 21578 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 19 0 8 0 263524314 889806848 19097 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21579] ppid=21571 vsize=868952 CPUtime=2.25
/proc/21574/task/21579/stat : 21579 (java) S 21571 21571 21452 0 -1 64 2472 0 0 0 213 12 0 0 16 0 8 0 263524314 889806848 19097 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21580] ppid=21571 vsize=868952 CPUtime=0
/proc/21574/task/21580/stat : 21580 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 20 0 8 0 263524314 889806848 19097 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21581] ppid=21571 vsize=868952 CPUtime=0
/proc/21574/task/21581/stat : 21581 (java) S 21571 21571 21452 0 -1 64 0 0 0 0 0 0 0 0 15 0 8 0 263524314 889806848 19097 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 1707.86
Current children cumulated vsize (KiB) 874304

[startup+1783.68 s]
/proc/loadavg: 2.05 2.03 1.91 3/102 21707
/proc/meminfo: memFree=796000/2055920 swapFree=4171264/4192956
[pid=21571] ppid=21569 vsize=5352 CPUtime=0
/proc/21571/stat : 21571 (cspfj.sh) S 21569 21571 21452 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 263524298 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 255236563780 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21571/statm: 1338 240 201 169 0 49 0
[pid=21574] ppid=21571 vsize=869084 CPUtime=1767.27
/proc/21574/stat : 21574 (java) R 21571 21571 21452 0 -1 0 21154 0 1 0 176641 86 0 0 25 0 8 0 263524299 889942016 19192 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4074234087 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/21574/statm: 217271 19192 1841 14 0 204532 0
[pid=21574/tid=21575] ppid=21571 vsize=869084 CPUtime=11.55
/proc/21574/task/21575/stat : 21575 (java) S 21571 21571 21452 0 -1 64 9734 0 0 0 1143 12 0 0 16 0 8 0 263524306 889942016 19192 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21576] ppid=21571 vsize=869084 CPUtime=0
/proc/21574/task/21576/stat : 21576 (java) S 21571 21571 21452 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 263524307 889942016 19192 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21577] ppid=21571 vsize=869084 CPUtime=0
/proc/21574/task/21577/stat : 21577 (java) S 21571 21571 21452 0 -1 64 7 0 0 0 0 0 0 0 16 0 8 0 263524307 889942016 19192 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21578] ppid=21571 vsize=869084 CPUtime=0
/proc/21574/task/21578/stat : 21578 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 19 0 8 0 263524314 889942016 19192 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21579] ppid=21571 vsize=869084 CPUtime=2.39
/proc/21574/task/21579/stat : 21579 (java) S 21571 21571 21452 0 -1 64 2567 0 0 0 226 13 0 0 15 0 8 0 263524314 889942016 19192 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21580] ppid=21571 vsize=869084 CPUtime=0
/proc/21574/task/21580/stat : 21580 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 20 0 8 0 263524314 889942016 19192 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21581] ppid=21571 vsize=869084 CPUtime=0
/proc/21574/task/21581/stat : 21581 (java) S 21571 21571 21452 0 -1 64 0 0 0 0 0 0 0 0 15 0 8 0 263524314 889942016 19192 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 1767.27
Current children cumulated vsize (KiB) 874436



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1816.74 s]
/proc/loadavg: 2.02 2.03 1.91 3/102 21707
/proc/meminfo: memFree=795680/2055920 swapFree=4171264/4192956
[pid=21571] ppid=21569 vsize=5352 CPUtime=0
/proc/21571/stat : 21571 (cspfj.sh) S 21569 21571 21452 0 -1 4194304 315 128 0 0 0 0 0 0 18 0 1 0 263524298 5480448 240 18446744073709551615 4194304 4889804 548682069312 18446744073709551615 255236563780 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21571/statm: 1338 240 201 169 0 49 0
[pid=21574] ppid=21571 vsize=869084 CPUtime=1800.02
/proc/21574/stat : 21574 (java) R 21571 21571 21452 0 -1 0 21218 0 1 0 179915 87 0 0 25 0 8 0 263524299 889942016 19216 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4074234175 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/21574/statm: 217271 19216 1841 14 0 204532 0
[pid=21574/tid=21575] ppid=21571 vsize=869084 CPUtime=11.83
/proc/21574/task/21575/stat : 21575 (java) S 21571 21571 21452 0 -1 64 9734 0 0 0 1171 12 0 0 16 0 8 0 263524306 889942016 19216 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21576] ppid=21571 vsize=869084 CPUtime=0
/proc/21574/task/21576/stat : 21576 (java) S 21571 21571 21452 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 263524307 889942016 19216 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21577] ppid=21571 vsize=869084 CPUtime=0
/proc/21574/task/21577/stat : 21577 (java) S 21571 21571 21452 0 -1 64 7 0 0 0 0 0 0 0 16 0 8 0 263524307 889942016 19216 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21578] ppid=21571 vsize=869084 CPUtime=0
/proc/21574/task/21578/stat : 21578 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 19 0 8 0 263524314 889942016 19216 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21579] ppid=21571 vsize=869084 CPUtime=2.42
/proc/21574/task/21579/stat : 21579 (java) S 21571 21571 21452 0 -1 64 2591 0 0 0 229 13 0 0 15 0 8 0 263524314 889942016 19216 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21580] ppid=21571 vsize=869084 CPUtime=0
/proc/21574/task/21580/stat : 21580 (java) S 21571 21571 21452 0 -1 64 1 0 0 0 0 0 0 0 20 0 8 0 263524314 889942016 19216 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=21574/tid=21581] ppid=21571 vsize=869084 CPUtime=0
/proc/21574/task/21581/stat : 21581 (java) S 21571 21571 21452 0 -1 64 0 0 0 0 0 0 0 0 15 0 8 0 263524314 889942016 19216 18446744073709551615 134512640 134570532 4294956528 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 1800.02
Current children cumulated vsize (KiB) 874436

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Child ended because it received signal 15 (SIGTERM)

!!! problem with CPU time !!!
wait4(...,&childrusage) returns 21571 and gives childrusage.ru_utime.tv_sec=0 childrusage.ru_utime.tv_usec=0 childrusage.ru_stime.tv_sec=0 childrusage.ru_stime.tv_usec=2999
CPU time returned by wait4() is 0.002999
while last known CPU time is 1800.02

Solver probably didn't/couldn't wait for its children
Using last known CPU time as value...

Real time (s): 1816.76
CPU time (s): 1800.02
CPU user time (s): 1799.15
CPU system time (s): 0.87
CPU usage (%): 99.0785
Max. virtual memory (cumulated for all children) (KiB): 874436

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0
system time used= 0.002999
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 443
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 17
involuntary context switches= 5

runsolver used 2.49862 s user time and 6.09507 s system time

The end



Launcher Data (download as text)




Begin job on node19 on Thu Jan 18 20:14:57 UTC 2007


IDJOB= 270891
IDBENCH= 4649
IDSOLVER= 91
FILE ID= node19/270891-1169151296

PBS_JOBID= 3582646

Free space on /tmp= 66559 MiB

SOLVER NAME= CSP4J - MAC 2007-01-16
BENCH NAME= HOME/pub/bench/CPAI06/pseudoSeries/uclid/ooo-burch-dill-4-accl.xml
COMMAND LINE= /tmp/evaluation/270891-1169151296/cspfj.sh  -competition /tmp/evaluation/270891-1169151296/unknown.xml
RUNSOLVER COMMAND LINE= runsolver  --timestamp  -w ROOT/results/node19/watcher-270891-1169151296 -o ROOT/results/node19/solver-270891-1169151296 -C 1800 -M 900  /tmp/evaluation/270891-1169151296/cspfj.sh  -competition /tmp/evaluation/270891-1169151296/unknown.xml

META MD5SUM SOLVER= 1cf1b9d4821c2ee5eee0dc0e02656faa
MD5SUM BENCH=  86c8ce99ba43ad8e61bc7c9612a8229b

RANDOM SEED= 328688766

TIME LIMIT= 1800 seconds

MEMORY LIMIT= 900 MiB


/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.260
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5914.62
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.260
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5586.94
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:      2055920 kB
MemFree:       1569112 kB
Buffers:         47144 kB
Cached:         331644 kB
SwapCached:      10172 kB
Active:         239120 kB
Inactive:       175296 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      2055920 kB
LowFree:       1569112 kB
SwapTotal:     4192956 kB
SwapFree:      4171264 kB
Dirty:            6768 kB
Writeback:           0 kB
Mapped:          44556 kB
Slab:            57328 kB
Committed_AS:  6715548 kB
PageTables:       2248 kB
VmallocTotal: 536870911 kB
VmallocUsed:    264952 kB
VmallocChunk: 536605679 kB
HugePages_Total:     0
HugePages_Free:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 66562 MiB



End job on node19 on Thu Jan 18 20:45:13 UTC 2007