Trace number 237532

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds, and are wall clock time (not CPU time).

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Abscon 109 ESAC? (TO) 1800.21 1816.84

General information on the benchmark

Namehaystacks/
haystacks_26.xml
MD5SUMb92463ea8cbb51f9a3234236cd8f2202
Bench Category2-ARY-INT (binary constraints in intension)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark31.5232
SatisfiableNO
(Un)Satisfiability was proved
Number of variables676
Number of constraints8475
Maximum constraint arity2
Maximum domain size26
Number of constraints which are defined in extension0
Number of constraints which are defined in intension8475
Global constraints used (with number of constraints)

Solver Data (download as text)

8.52	   
8.52	Abscon 109 (November, 2006)
8.52	   
8.52	   configuration
8.52	     name=esac.xml
9.50	   
9.50	   instance
9.50	     name=/tmp/evaluation/237532-1168349095/unknown.xml
9.53	   
9.53	   domains being loaded...
9.53	   
9.53	   variables being loaded...
9.63	     nbVariables=676  nbDomainTypes=1  minDomainSize=26  maxDomainSize=26
9.63	       =>  wckTime=1.106  cpuTime=0.12  mem=7M862
9.63	   
9.63	   predicates being loaded...
9.63	   
9.63	   constraints being loaded...
10.80	     nbConstraints=8475  nbConvertedConstraints=8475  nbConvertConstraintsChecks=2704  nbExtensionStructures=4
10.80	     nbSharedExtensionStructures=8471  nbConflictsStructures=4  nbSharedConflictsStructures=8471
10.80	     nbSharedBinaryRepresentations=155  maxDegree=26  maxArity=2
10.80	       =>  wckTime=2.278  cpuTime=1.07  mem=12M657
10.88	   
10.88	   solver IterativeSystematicSolver being built... 
11.08	       =>  wckTime=2.558  cpuTime=1.22  mem=22M356
11.73	   
11.73	   preprocessing
11.73	     nbConstraintChecks=214057  nbPropagations=12075  nbRevisions=12717  nbUselessRevisions=1810  nbSingletonTests=773
11.73	     nbEffectiveSingletonTests=0  nbESACBranches=2  nbRemovedValues=0  detectedInconsistency=no
11.73	       =>  wckTime=0.644  cpuTime=0.62  mem=22M356
11.81	   
11.81	   search
11.81	     run=0  nbConstraintChecks=34318  nbPropagations=13786  nbRevisions=14485  nbUselessRevisions=1896  nbAssignments=90
11.81	     nbFailedAssignments=10  nbBacktracks=10  nbVisitedNodes=111  nbRestartNogoods=0  mapSize=10  nbInferences=0
11.81	     nbTooLargeKeys=0
11.81	       =>  wckTime=0.725  cpuTime=0.69  mem=22M356
11.85	   
11.85	   search
11.85	     run=1  nbConstraintChecks=68332  nbPropagations=29327  nbRevisions=30769  nbUselessRevisions=3862
11.85	     nbAssignments=190  nbFailedAssignments=24  nbBacktracks=25  nbVisitedNodes=239  nbRestartNogoods=4  mapSize=25
11.85	     nbInferences=0  nbTooLargeKeys=0
11.85	       =>  wckTime=0.769  cpuTime=0.73  mem=22M356
11.90	   
11.90	   search
11.90	     run=2  nbConstraintChecks=102495  nbPropagations=46662  nbRevisions=48901  nbUselessRevisions=5903
11.90	     nbAssignments=299  nbFailedAssignments=42  nbBacktracks=47  nbVisitedNodes=384  nbRestartNogoods=7  mapSize=45
11.90	     nbInferences=0  nbTooLargeKeys=0
11.90	       =>  wckTime=0.815  cpuTime=0.78  mem=23M351
11.95	   
11.95	   search
11.95	     run=3  nbConstraintChecks=136973  nbPropagations=65885  nbRevisions=68974  nbUselessRevisions=8020
11.95	     nbAssignments=428  nbFailedAssignments=68  nbBacktracks=80  nbVisitedNodes=566  nbRestartNogoods=10  mapSize=77
11.95	     nbInferences=1  nbTooLargeKeys=0
11.95	       =>  wckTime=0.865  cpuTime=0.83  mem=23M351
12.01	   
12.01	   search
12.01	     run=4  nbConstraintChecks=171853  nbPropagations=87144  nbRevisions=91131  nbUselessRevisions=10215
12.01	     nbAssignments=589  nbFailedAssignments=105  nbBacktracks=131  nbVisitedNodes=806  nbRestartNogoods=13  mapSize=128
12.01	     nbInferences=6  nbTooLargeKeys=0
12.01	       =>  wckTime=0.921  cpuTime=0.88  mem=23M351
12.08	   
12.08	   search
12.08	     run=5  nbConstraintChecks=207413  nbPropagations=110668  nbRevisions=115576  nbUselessRevisions=12487
12.08	     nbAssignments=798  nbFailedAssignments=154  nbBacktracks=206  nbVisitedNodes=1130  nbRestartNogoods=17  mapSize=203
12.08	     nbInferences=23  nbTooLargeKeys=0
12.08	       =>  wckTime=0.994  cpuTime=0.95  mem=24M345
12.16	   
12.16	   search
12.16	     run=6  nbConstraintChecks=243809  nbPropagations=136777  nbRevisions=142622  nbUselessRevisions=14840
12.16	     nbAssignments=1078  nbFailedAssignments=229  nbBacktracks=320  nbVisitedNodes=1582  nbRestartNogoods=19
12.16	     mapSize=317  nbInferences=45  nbTooLargeKeys=0
12.16	       =>  wckTime=1.08  cpuTime=1.03  mem=24M345
12.29	   
12.29	   search
12.29	     run=7  nbConstraintChecks=281531  nbPropagations=165971  nbRevisions=172731  nbUselessRevisions=17277
12.29	     nbAssignments=1467  nbFailedAssignments=336  nbBacktracks=490  nbVisitedNodes=2227  nbRestartNogoods=25
12.29	     mapSize=487  nbInferences=87  nbTooLargeKeys=0
12.29	       =>  wckTime=1.204  cpuTime=1.14  mem=25M340
12.45	   
12.46	   search
12.46	     run=8  nbConstraintChecks=321338  nbPropagations=199033  nbRevisions=206609  nbUselessRevisions=19802
12.46	     nbAssignments=2026  nbFailedAssignments=487  nbBacktracks=746  nbVisitedNodes=3171  nbRestartNogoods=33
12.46	     mapSize=743  nbInferences=170  nbTooLargeKeys=0
12.46	       =>  wckTime=1.371  cpuTime=1.3  mem=27M329
12.62	   
12.62	   search
12.62	     run=9  nbConstraintChecks=369058  nbPropagations=236599  nbRevisions=245143  nbUselessRevisions=22517
12.62	     nbAssignments=2708  nbFailedAssignments=736  nbBacktracks=1131  nbVisitedNodes=4367  nbRestartNogoods=40
12.62	     mapSize=1102  nbInferences=186  nbTooLargeKeys=0
12.62	       =>  wckTime=1.539  cpuTime=1.46  mem=28M324
12.87	   
12.87	   search
12.87	     run=10  nbConstraintChecks=433043  nbPropagations=281309  nbRevisions=290792  nbUselessRevisions=25689
12.87	     nbAssignments=3876  nbFailedAssignments=1045  nbBacktracks=1707  nbVisitedNodes=6413  nbRestartNogoods=41
12.87	     mapSize=1643  nbInferences=446  nbTooLargeKeys=0
12.87	       =>  wckTime=1.785  cpuTime=1.7  mem=29M319
13.30	   
13.30	   search
13.30	     run=11  nbConstraintChecks=554518  nbPropagations=338266  nbRevisions=348422  nbUselessRevisions=29391
13.30	     nbAssignments=5717  nbFailedAssignments=1504  nbBacktracks=2571  nbVisitedNodes=9502  nbRestartNogoods=47
13.30	     mapSize=2467  nbInferences=823  nbTooLargeKeys=0
13.30	       =>  wckTime=2.212  cpuTime=2.13  mem=32M303
14.08	   
14.08	   search
14.08	     run=12  nbConstraintChecks=710710  nbPropagations=414771  nbRevisions=424096  nbUselessRevisions=33661
14.08	     nbAssignments=8826  nbFailedAssignments=2106  nbBacktracks=3869  nbVisitedNodes=14745  nbRestartNogoods=58
14.08	     mapSize=3741  nbInferences=1847  nbTooLargeKeys=0
14.08	       =>  wckTime=2.992  cpuTime=2.89  mem=40M260
15.39	   
15.39	   search
15.39	     run=13  nbConstraintChecks=819760  nbPropagations=517769  nbRevisions=522170  nbUselessRevisions=38194
15.39	     nbAssignments=13821  nbFailedAssignments=2870  nbBacktracks=5815  nbVisitedNodes=23463  nbRestartNogoods=65
15.39	     mapSize=5687  nbInferences=4042  nbTooLargeKeys=0
15.39	       =>  wckTime=4.304  cpuTime=4.15  mem=25M308
16.56	   
16.56	   search
16.56	     run=14  nbConstraintChecks=1105228  nbPropagations=655358  nbRevisions=655141  nbUselessRevisions=44760
16.56	     nbAssignments=19610  nbFailedAssignments=4594  nbBacktracks=8734  nbVisitedNodes=33621  nbRestartNogoods=74
16.56	     mapSize=8471  nbInferences=5054  nbTooLargeKeys=0
16.56	       =>  wckTime=5.478  cpuTime=5.31  mem=30M316
19.78	   
19.78	   search
19.78	     run=15  nbConstraintChecks=1514621  nbPropagations=868932  nbRevisions=852312  nbUselessRevisions=53258
19.78	     nbAssignments=32396  nbFailedAssignments=5955  nbBacktracks=13113  nbVisitedNodes=55973  nbRestartNogoods=81
19.78	     mapSize=12745  nbInferences=11942  nbTooLargeKeys=0
19.78	       =>  wckTime=8.694  cpuTime=8.5  mem=68M378
24.83	   
24.83	   search
24.83	     run=16  nbConstraintChecks=2075809  nbPropagations=1196312  nbRevisions=1143545  nbUselessRevisions=63548
24.83	     nbAssignments=52001  nbFailedAssignments=7880  nbBacktracks=19682  nbVisitedNodes=89805  nbRestartNogoods=106
24.83	     mapSize=19252  nbInferences=22630  nbTooLargeKeys=0
24.83	       =>  wckTime=13.742  cpuTime=13.37  mem=66M916
32.74	   
32.74	   search
32.74	     run=17  nbConstraintChecks=2792076  nbPropagations=1701485  nbRevisions=1576951  nbUselessRevisions=75362
32.74	     nbAssignments=83108  nbFailedAssignments=10372  nbBacktracks=29534  nbVisitedNodes=144344  nbRestartNogoods=124
32.74	     mapSize=29043  nbInferences=41182  nbTooLargeKeys=0
32.74	       =>  wckTime=21.655  cpuTime=21.19  mem=97M832
45.61	   
45.61	   search
45.61	     run=18  nbConstraintChecks=4341955  nbPropagations=2529686  nbRevisions=2270068  nbUselessRevisions=91679
45.61	     nbAssignments=135185  nbFailedAssignments=13226  nbBacktracks=44312  nbVisitedNodes=235165  nbRestartNogoods=143
45.61	     mapSize=43675  nbInferences=74856  nbTooLargeKeys=0
45.61	       =>  wckTime=34.531  cpuTime=33.92  mem=151M681
66.29	   
66.29	   search
66.29	     run=19  nbConstraintChecks=6890615  nbPropagations=3878643  nbRevisions=3386002  nbUselessRevisions=118588
66.29	     nbAssignments=217428  nbFailedAssignments=17341  nbBacktracks=66480  nbVisitedNodes=378444  nbRestartNogoods=175
66.29	     mapSize=65722  nbInferences=129172  nbTooLargeKeys=0
66.29	       =>  wckTime=55.206  cpuTime=54.36  mem=224M788
96.04	   
96.04	   search
96.04	     run=20  nbConstraintChecks=10785458  nbPropagations=6009746  nbRevisions=5145115  nbUselessRevisions=165496
96.04	     nbAssignments=339799  nbFailedAssignments=24062  nbBacktracks=99732  nbVisitedNodes=593851  nbRestartNogoods=195
96.04	     mapSize=98839  nbInferences=210128  nbTooLargeKeys=0
96.04	       =>  wckTime=84.951  cpuTime=83.43  mem=320M703
145.30	   
145.30	   search
145.30	     run=21  nbConstraintChecks=16358437  nbPropagations=9400492  nbRevisions=7899682  nbUselessRevisions=227461
145.30	     nbAssignments=537932  nbFailedAssignments=30621  nbBacktracks=149611  nbVisitedNodes=942079  nbRestartNogoods=229
145.30	     mapSize=148617  nbInferences=349386  nbTooLargeKeys=0
145.30	       =>  wckTime=134.216  cpuTime=132.03  mem=469M652
177.62	   
177.62	   INFO : Stopping hyperNogood (mapSize=184972, nbTooLargekeys=0, mem=600M670)
185.01	   
185.01	   search
185.01	     run=22  nbConstraintChecks=20839704  nbPropagations=13945118  nbRevisions=11636280  nbUselessRevisions=303586
185.01	     nbAssignments=740689  nbFailedAssignments=63762  nbBacktracks=224429  nbVisitedNodes=1303637  nbRestartNogoods=260
185.01	       =>  wckTime=173.931  cpuTime=171.3  mem=600M670
207.14	   
207.14	   search
207.14	     run=23  nbConstraintChecks=24842299  nbPropagations=19379415  nbRevisions=16316008  nbUselessRevisions=386527
207.14	     nbAssignments=931894  nbFailedAssignments=141470  nbBacktracks=336656  nbVisitedNodes=1649028  nbRestartNogoods=298
207.14	       =>  wckTime=196.058  cpuTime=193.34  mem=600M670
242.36	   
242.36	   search
242.36	     run=24  nbConstraintChecks=37552053  nbPropagations=26401688  nbRevisions=22923768  nbUselessRevisions=616391
242.36	     nbAssignments=1221384  nbFailedAssignments=261243  nbBacktracks=504997  nbVisitedNodes=2174528
242.36	     nbRestartNogoods=322
242.36	       =>  wckTime=231.274  cpuTime=228.42  mem=600M670
290.00	   
290.00	   search
290.00	     run=25  nbConstraintChecks=43298890  nbPropagations=35366639  nbRevisions=31482261  nbUselessRevisions=855193
290.00	     nbAssignments=1648354  nbFailedAssignments=434210  nbBacktracks=757508  nbVisitedNodes=2946136
290.00	     nbRestartNogoods=366
290.00	       =>  wckTime=278.918  cpuTime=275.88  mem=600M670
371.83	   
371.83	   search
371.83	     run=26  nbConstraintChecks=79931791  nbPropagations=48247757  nbRevisions=45158090  nbUselessRevisions=1718149
371.83	     nbAssignments=2298340  nbFailedAssignments=704568  nbBacktracks=1136275  nbVisitedNodes=4129198
371.83	     nbRestartNogoods=400
371.83	       =>  wckTime=360.748  cpuTime=357.39  mem=600M670
475.06	   
475.06	   search
475.06	     run=27  nbConstraintChecks=91378539  nbPropagations=65448833  nbRevisions=63162028  nbUselessRevisions=2589274
475.06	     nbAssignments=3260211  nbFailedAssignments=1096947  nbBacktracks=1704426  nbVisitedNodes=5874688
475.06	     nbRestartNogoods=442
475.06	       =>  wckTime=463.974  cpuTime=460.21  mem=600M670
703.96	   
703.96	   search
703.96	     run=28  nbConstraintChecks=354079501  nbPropagations=94864308  nbRevisions=107597286  nbUselessRevisions=8317848
703.96	     nbAssignments=4774527  nbFailedAssignments=1758040  nbBacktracks=2556652  nbVisitedNodes=8680672
703.96	     nbRestartNogoods=471
703.96	       =>  wckTime=692.874  cpuTime=688.23  mem=600M670
975.11	   
975.11	   search
975.11	     run=29  nbConstraintChecks=514881431  nbPropagations=137717490  nbRevisions=171654388  nbUselessRevisions=16042889
975.11	     nbAssignments=6978435  nbFailedAssignments=2682461  nbBacktracks=3834992  nbVisitedNodes=12689099
975.11	     nbRestartNogoods=508
975.11	       =>  wckTime=964.026  cpuTime=958.32  mem=600M670
1309.60	   
1309.60	   search
1309.60	     run=30  nbConstraintChecks=554392905  nbPropagations=195150533  nbRevisions=250409338  nbUselessRevisions=23806003
1309.60	     nbAssignments=10225095  nbFailedAssignments=4010260  nbBacktracks=5752502  nbVisitedNodes=18589632
1309.60	     nbRestartNogoods=536
1309.60	       =>  wckTime=1298.516  cpuTime=1291.51  mem=600M670
1796.42	   
1796.42	   search
1796.42	     run=31  nbConstraintChecks=611082970  nbPropagations=274376158  nbRevisions=351093444  nbUselessRevisions=31648031
1796.42	     nbAssignments=15091613  nbFailedAssignments=5999511  nbBacktracks=8628767  nbVisitedNodes=27433547
1796.42	     nbRestartNogoods=567
1796.42	       =>  wckTime=1785.336  cpuTime=1776.44  mem=600M670

Verifier Data (download as text)

ERROR: Unexpected answer ! (SAT/UNSAT expected)
Got answer:

Watcher Data (download as text)

runsolver version 3.1.3 (c) roussel@cril.univ-artois.fr

command line: runsolver --timestamp -w ROOT/results/node76/watcher-237532-1168349095 -o ROOT/results/node76/solver-237532-1168349095 -C 1800 -M 900 java -Xms756M -Xmx756M -XX:PermSize=10m -XX:MaxPermSize=10m -jar abscon109.jar esac.xml 1 XSax /tmp/evaluation/237532-1168349095/unknown.xml 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 921600 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 972800 KiB
Current StackSize limit: 10240 KiB

/proc/loadavg: 4.31 4.99 5.04 4/90 686
/proc/meminfo: memFree=14864/2055888 swapFree=4095388/4096564
[pid=685] ppid=683 vsize=18544 CPUtime=0
/proc/685/stat : 685 (runsolver) D 683 685 31949 0 -1 4194368 18 0 0 0 0 0 0 0 20 0 1 0 183331160 18989056 280 18446744073709551615 4194304 4267372 548682068960 18446744073709551615 206843737111 0 0 4096 24578 18446744071563611212 0 0 17 1 0 0
/proc/685/statm: 4636 280 245 17 0 2626 0

[startup+0.106771 s]
/proc/loadavg: 4.31 4.99 5.04 4/90 686
/proc/meminfo: memFree=14864/2055888 swapFree=4095388/4096564
[pid=685] ppid=683 vsize=18544 CPUtime=0
/proc/685/stat : 685 (runsolver) D 683 685 31949 0 -1 4194368 18 0 0 0 0 0 0 0 20 0 1 0 183331160 18989056 280 18446744073709551615 4194304 4267372 548682068960 18446744073709551615 206843737111 0 0 4096 24578 18446744071563611212 0 0 17 1 0 0
/proc/685/statm: 4636 280 245 17 0 2626 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18544

[startup+0.514851 s]
/proc/loadavg: 4.31 4.99 5.04 4/90 686
/proc/meminfo: memFree=14864/2055888 swapFree=4095388/4096564
[pid=685] ppid=683 vsize=18544 CPUtime=0
/proc/685/stat : 685 (runsolver) D 683 685 31949 0 -1 4194368 18 0 0 0 0 0 0 0 20 0 1 0 183331160 18989056 280 18446744073709551615 4194304 4267372 548682068960 18446744073709551615 206843737111 0 0 4096 24578 18446744071563611212 0 0 17 1 0 0
/proc/685/statm: 4636 280 245 17 0 2626 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18544

[startup+1.33601 s]
/proc/loadavg: 4.21 4.96 5.03 2/89 686
/proc/meminfo: memFree=16152/2055888 swapFree=4095388/4096564
[pid=685] ppid=683 vsize=18544 CPUtime=0
/proc/685/stat : 685 (runsolver) D 683 685 31949 0 -1 4194368 18 0 0 0 0 0 0 0 18 0 1 0 183331160 18989056 280 18446744073709551615 4194304 4267372 548682068960 18446744073709551615 206843737111 0 0 4096 24578 18446744071563611212 0 0 17 1 0 0
/proc/685/statm: 4636 280 245 17 0 2626 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18544

[startup+2.97333 s]
/proc/loadavg: 4.21 4.96 5.03 2/88 686
/proc/meminfo: memFree=15904/2055888 swapFree=4095388/4096564
[pid=685] ppid=683 vsize=296 CPUtime=0
/proc/685/stat : 685 (java) D 683 685 31949 0 -1 0 47 0 1 0 0 0 0 0 18 0 1 0 183331160 303104 31 18446744073709551615 134512640 134570532 4294956624 18446744073709551615 4822167 0 0 4096 0 18446744071563481621 0 0 17 1 0 0
/proc/685/statm: 74 31 25 14 0 3 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 296

[startup+6.25397 s]
/proc/loadavg: 4.11 4.92 5.02 2/86 686
/proc/meminfo: memFree=15280/2055888 swapFree=4095388/4096564
[pid=685] ppid=683 vsize=9964 CPUtime=0
/proc/685/stat : 685 (java) D 683 685 31949 0 -1 0 674 0 8 0 0 0 0 0 18 0 1 0 183331160 10203136 448 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4822167 0 0 0 7368 18446744071563481621 0 0 17 1 0 0
/proc/685/statm: 2491 448 366 14 0 1142 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9964

[startup+12.7262 s]
/proc/loadavg: 4.26 4.94 5.02 3/94 694
/proc/meminfo: memFree=15024/2055888 swapFree=4095388/4096564
[pid=685] ppid=683 vsize=916400 CPUtime=3.46
/proc/685/stat : 685 (java) R 683 685 31949 0 -1 0 19915 0 15 0 306 40 0 0 25 0 8 0 183331160 938393600 19664 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4073996264 0 4 0 23759 18446744073709551615 0 0 17 1 0 0
/proc/685/statm: 229100 19664 2546 14 0 215569 0
[pid=685/tid=687] ppid=683 vsize=916400 CPUtime=0.13
/proc/685/task/687/stat : 687 (java) S 683 685 31949 0 -1 64 2791 0 0 0 13 0 0 0 17 0 8 0 183331898 938393600 19664 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=688] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/688/stat : 688 (java) S 683 685 31949 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 19664 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=689] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/689/stat : 689 (java) S 683 685 31949 0 -1 64 5 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 19664 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=690] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/690/stat : 690 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 18 0 8 0 183331990 938393600 19664 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=691] ppid=683 vsize=916400 CPUtime=0.28
/proc/685/task/691/stat : 691 (java) S 683 685 31949 0 -1 64 498 0 0 0 27 1 0 0 15 0 8 0 183331990 938393600 19664 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=692] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/692/stat : 692 (java) S 683 685 31949 0 -1 64 0 0 0 0 0 0 0 0 19 0 8 0 183331990 938393600 19664 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=693] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/693/stat : 693 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 183331990 938393600 19664 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 3.46
Current children cumulated vsize (KiB) 916400

[startup+25.5758 s]
/proc/loadavg: 4.92 5.06 5.06 3/101 702
/proc/meminfo: memFree=15224/2055888 swapFree=4095388/4096564
[pid=685] ppid=683 vsize=916400 CPUtime=16.16
/proc/685/stat : 685 (java) R 683 685 31949 0 -1 0 33793 0 15 0 1570 46 0 0 25 0 8 0 183331160 938393600 33535 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4072906241 0 4 0 23759 18446744073709551615 0 0 17 1 0 0
/proc/685/statm: 229100 33535 2546 14 0 215569 0
[pid=685/tid=687] ppid=683 vsize=916400 CPUtime=0.26
/proc/685/task/687/stat : 687 (java) S 683 685 31949 0 -1 64 16658 0 0 0 20 6 0 0 17 0 8 0 183331898 938393600 33535 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=688] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/688/stat : 688 (java) S 683 685 31949 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 33535 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=689] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/689/stat : 689 (java) S 683 685 31949 0 -1 64 5 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 33535 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=690] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/690/stat : 690 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 18 0 8 0 183331990 938393600 33535 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=691] ppid=683 vsize=916400 CPUtime=0.29
/proc/685/task/691/stat : 691 (java) S 683 685 31949 0 -1 64 502 0 0 0 28 1 0 0 16 0 8 0 183331990 938393600 33535 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=692] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/692/stat : 692 (java) S 683 685 31949 0 -1 64 0 0 0 0 0 0 0 0 19 0 8 0 183331990 938393600 33535 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=693] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/693/stat : 693 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 183331990 938393600 33535 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 16.16
Current children cumulated vsize (KiB) 916400

[startup+51.2768 s]
/proc/loadavg: 5.03 5.08 5.07 2/89 703
/proc/meminfo: memFree=50200/2055888 swapFree=4095388/4096564
[pid=685] ppid=683 vsize=916400 CPUtime=41.81
/proc/685/stat : 685 (java) R 683 685 31949 0 -1 0 57368 0 15 0 4123 58 0 0 25 0 8 0 183331160 938393600 57104 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4074052935 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/685/statm: 229100 57104 2546 14 0 215569 0
[pid=685/tid=687] ppid=683 vsize=916400 CPUtime=0.49
/proc/685/task/687/stat : 687 (java) S 683 685 31949 0 -1 64 40225 0 0 0 35 14 0 0 16 0 8 0 183331898 938393600 57104 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=688] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/688/stat : 688 (java) S 683 685 31949 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 57104 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=689] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/689/stat : 689 (java) S 683 685 31949 0 -1 64 5 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 57104 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=690] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/690/stat : 690 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 18 0 8 0 183331990 938393600 57104 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=691] ppid=683 vsize=916400 CPUtime=0.29
/proc/685/task/691/stat : 691 (java) S 683 685 31949 0 -1 64 504 0 0 0 28 1 0 0 16 0 8 0 183331990 938393600 57104 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=692] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/692/stat : 692 (java) S 683 685 31949 0 -1 64 0 0 0 0 0 0 0 0 19 0 8 0 183331990 938393600 57104 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=693] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/693/stat : 693 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 183331990 938393600 57104 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 41.81
Current children cumulated vsize (KiB) 916400

[startup+102.542 s]
/proc/loadavg: 3.57 4.69 4.94 3/93 773
/proc/meminfo: memFree=1508208/2055888 swapFree=4095388/4096564
[pid=685] ppid=683 vsize=916400 CPUtime=92.58
/proc/685/stat : 685 (java) R 683 685 31949 0 -1 0 98530 0 15 0 9178 80 0 0 25 0 8 0 183331160 938393600 98254 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4073996276 0 4 0 23759 18446744073709551615 0 0 17 1 0 0
/proc/685/statm: 229100 98254 2546 14 0 215569 0
[pid=685/tid=687] ppid=683 vsize=916400 CPUtime=0.95
/proc/685/task/687/stat : 687 (java) S 683 685 31949 0 -1 64 81375 0 0 0 66 29 0 0 16 0 8 0 183331898 938393600 98254 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=688] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/688/stat : 688 (java) S 683 685 31949 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 98254 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=689] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/689/stat : 689 (java) S 683 685 31949 0 -1 64 5 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 98254 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=690] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/690/stat : 690 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 18 0 8 0 183331990 938393600 98254 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=691] ppid=683 vsize=916400 CPUtime=0.29
/proc/685/task/691/stat : 691 (java) S 683 685 31949 0 -1 64 504 0 0 0 28 1 0 0 16 0 8 0 183331990 938393600 98254 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=692] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/692/stat : 692 (java) S 683 685 31949 0 -1 64 0 0 0 0 0 0 0 0 19 0 8 0 183331990 938393600 98254 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=693] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/693/stat : 693 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 183331990 938393600 98254 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 92.58
Current children cumulated vsize (KiB) 916400

[startup+162.58 s]
/proc/loadavg: 2.64 4.22 4.75 3/93 773
/proc/meminfo: memFree=1256952/2055888 swapFree=4095388/4096564
[pid=685] ppid=683 vsize=916400 CPUtime=152.38
/proc/685/stat : 685 (java) R 683 685 31949 0 -1 0 147052 0 15 0 15139 99 0 0 25 0 8 0 183331160 938393600 146761 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4074047728 0 4 0 23759 18446744073709551615 0 0 17 1 0 0
/proc/685/statm: 229100 146761 2546 14 0 215569 0
[pid=685/tid=687] ppid=683 vsize=916400 CPUtime=1.53
/proc/685/task/687/stat : 687 (java) S 683 685 31949 0 -1 64 129881 0 0 0 106 47 0 0 16 0 8 0 183331898 938393600 146761 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=688] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/688/stat : 688 (java) S 683 685 31949 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 146761 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=689] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/689/stat : 689 (java) S 683 685 31949 0 -1 64 5 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 146761 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=690] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/690/stat : 690 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 18 0 8 0 183331990 938393600 146761 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=691] ppid=683 vsize=916400 CPUtime=0.29
/proc/685/task/691/stat : 691 (java) S 683 685 31949 0 -1 64 505 0 0 0 28 1 0 0 16 0 8 0 183331990 938393600 146761 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=692] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/692/stat : 692 (java) S 683 685 31949 0 -1 64 0 0 0 0 0 0 0 0 19 0 8 0 183331990 938393600 146761 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=693] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/693/stat : 693 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 183331990 938393600 146761 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 152.38
Current children cumulated vsize (KiB) 916400

[startup+222.595 s]
/proc/loadavg: 2.30 3.83 4.58 3/93 773
/proc/meminfo: memFree=1210424/2055888 swapFree=4095388/4096564
[pid=685] ppid=683 vsize=916400 CPUtime=212.16
/proc/685/stat : 685 (java) R 683 685 31949 0 -1 0 155875 0 15 0 21114 102 0 0 25 0 8 0 183331160 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4074035176 0 4 0 23759 18446744073709551615 0 0 17 1 0 0
/proc/685/statm: 229100 155580 2546 14 0 215569 0
[pid=685/tid=687] ppid=683 vsize=916400 CPUtime=1.63
/proc/685/task/687/stat : 687 (java) S 683 685 31949 0 -1 64 138700 0 0 0 113 50 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=688] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/688/stat : 688 (java) S 683 685 31949 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=689] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/689/stat : 689 (java) S 683 685 31949 0 -1 64 5 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=690] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/690/stat : 690 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 18 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=691] ppid=683 vsize=916400 CPUtime=0.29
/proc/685/task/691/stat : 691 (java) S 683 685 31949 0 -1 64 505 0 0 0 28 1 0 0 16 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=692] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/692/stat : 692 (java) S 683 685 31949 0 -1 64 0 0 0 0 0 0 0 0 19 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=693] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/693/stat : 693 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 212.16
Current children cumulated vsize (KiB) 916400

[startup+282.698 s]

################
# More data... #
################

Current children cumulated CPU time (s) 1408.81
Current children cumulated vsize (KiB) 916400

[startup+1484.02 s]
/proc/loadavg: 2.05 2.03 2.65 3/93 773
/proc/meminfo: memFree=1209848/2055888 swapFree=4095388/4096564
[pid=685] ppid=683 vsize=916400 CPUtime=1468.68
/proc/685/stat : 685 (java) R 683 685 31949 0 -1 0 155875 0 15 0 146766 102 0 0 25 0 8 0 183331160 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4073987872 0 4 0 23759 18446744073709551615 0 0 17 1 0 0
/proc/685/statm: 229100 155580 2546 14 0 215569 0
[pid=685/tid=687] ppid=683 vsize=916400 CPUtime=1.63
/proc/685/task/687/stat : 687 (java) S 683 685 31949 0 -1 64 138700 0 0 0 113 50 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=688] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/688/stat : 688 (java) S 683 685 31949 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=689] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/689/stat : 689 (java) S 683 685 31949 0 -1 64 5 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=690] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/690/stat : 690 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 18 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=691] ppid=683 vsize=916400 CPUtime=0.29
/proc/685/task/691/stat : 691 (java) S 683 685 31949 0 -1 64 505 0 0 0 28 1 0 0 16 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=692] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/692/stat : 692 (java) S 683 685 31949 0 -1 64 0 0 0 0 0 0 0 0 19 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=693] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/693/stat : 693 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 1468.68
Current children cumulated vsize (KiB) 916400

[startup+1544.12 s]
/proc/loadavg: 2.08 2.04 2.61 3/93 773
/proc/meminfo: memFree=1209848/2055888 swapFree=4095388/4096564
[pid=685] ppid=683 vsize=916400 CPUtime=1528.55
/proc/685/stat : 685 (java) R 683 685 31949 0 -1 0 155875 0 15 0 152753 102 0 0 25 0 8 0 183331160 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4074003455 0 4 0 23759 18446744073709551615 0 0 17 1 0 0
/proc/685/statm: 229100 155580 2546 14 0 215569 0
[pid=685/tid=687] ppid=683 vsize=916400 CPUtime=1.63
/proc/685/task/687/stat : 687 (java) S 683 685 31949 0 -1 64 138700 0 0 0 113 50 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=688] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/688/stat : 688 (java) S 683 685 31949 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=689] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/689/stat : 689 (java) S 683 685 31949 0 -1 64 5 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=690] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/690/stat : 690 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 18 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=691] ppid=683 vsize=916400 CPUtime=0.29
/proc/685/task/691/stat : 691 (java) S 683 685 31949 0 -1 64 505 0 0 0 28 1 0 0 16 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=692] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/692/stat : 692 (java) S 683 685 31949 0 -1 64 0 0 0 0 0 0 0 0 19 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=693] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/693/stat : 693 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 1528.55
Current children cumulated vsize (KiB) 916400

[startup+1604.13 s]
/proc/loadavg: 2.03 2.03 2.57 3/93 773
/proc/meminfo: memFree=1209848/2055888 swapFree=4095388/4096564
[pid=685] ppid=683 vsize=916400 CPUtime=1588.32
/proc/685/stat : 685 (java) R 683 685 31949 0 -1 0 155875 0 15 0 158730 102 0 0 25 0 8 0 183331160 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4074033113 0 4 0 23759 18446744073709551615 0 0 17 1 0 0
/proc/685/statm: 229100 155580 2546 14 0 215569 0
[pid=685/tid=687] ppid=683 vsize=916400 CPUtime=1.63
/proc/685/task/687/stat : 687 (java) S 683 685 31949 0 -1 64 138700 0 0 0 113 50 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=688] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/688/stat : 688 (java) S 683 685 31949 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=689] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/689/stat : 689 (java) S 683 685 31949 0 -1 64 5 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=690] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/690/stat : 690 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 18 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=691] ppid=683 vsize=916400 CPUtime=0.29
/proc/685/task/691/stat : 691 (java) S 683 685 31949 0 -1 64 505 0 0 0 28 1 0 0 16 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=692] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/692/stat : 692 (java) S 683 685 31949 0 -1 64 0 0 0 0 0 0 0 0 19 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=693] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/693/stat : 693 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 1588.32
Current children cumulated vsize (KiB) 916400

[startup+1664.23 s]
/proc/loadavg: 2.01 2.02 2.53 3/93 773
/proc/meminfo: memFree=1209784/2055888 swapFree=4095388/4096564
[pid=685] ppid=683 vsize=916400 CPUtime=1648.18
/proc/685/stat : 685 (java) R 683 685 31949 0 -1 0 155875 0 15 0 164716 102 0 0 25 0 8 0 183331160 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4074035178 0 4 0 23759 18446744073709551615 0 0 17 1 0 0
/proc/685/statm: 229100 155580 2546 14 0 215569 0
[pid=685/tid=687] ppid=683 vsize=916400 CPUtime=1.63
/proc/685/task/687/stat : 687 (java) S 683 685 31949 0 -1 64 138700 0 0 0 113 50 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=688] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/688/stat : 688 (java) S 683 685 31949 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=689] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/689/stat : 689 (java) S 683 685 31949 0 -1 64 5 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=690] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/690/stat : 690 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 18 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=691] ppid=683 vsize=916400 CPUtime=0.29
/proc/685/task/691/stat : 691 (java) S 683 685 31949 0 -1 64 505 0 0 0 28 1 0 0 16 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=692] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/692/stat : 692 (java) S 683 685 31949 0 -1 64 0 0 0 0 0 0 0 0 19 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=693] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/693/stat : 693 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 1648.18
Current children cumulated vsize (KiB) 916400

[startup+1724.32 s]
/proc/loadavg: 2.00 2.02 2.50 3/93 773
/proc/meminfo: memFree=1209784/2055888 swapFree=4095388/4096564
[pid=685] ppid=683 vsize=916400 CPUtime=1708.04
/proc/685/stat : 685 (java) R 683 685 31949 0 -1 0 155875 0 15 0 170702 102 0 0 25 0 8 0 183331160 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4073977356 0 4 0 23759 18446744073709551615 0 0 17 1 0 0
/proc/685/statm: 229100 155580 2546 14 0 215569 0
[pid=685/tid=687] ppid=683 vsize=916400 CPUtime=1.63
/proc/685/task/687/stat : 687 (java) S 683 685 31949 0 -1 64 138700 0 0 0 113 50 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=688] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/688/stat : 688 (java) S 683 685 31949 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=689] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/689/stat : 689 (java) S 683 685 31949 0 -1 64 5 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=690] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/690/stat : 690 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 18 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=691] ppid=683 vsize=916400 CPUtime=0.29
/proc/685/task/691/stat : 691 (java) S 683 685 31949 0 -1 64 505 0 0 0 28 1 0 0 16 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=692] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/692/stat : 692 (java) S 683 685 31949 0 -1 64 0 0 0 0 0 0 0 0 19 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=693] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/693/stat : 693 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 1708.04
Current children cumulated vsize (KiB) 916400

[startup+1784.41 s]
/proc/loadavg: 2.00 2.01 2.46 3/93 773
/proc/meminfo: memFree=1209784/2055888 swapFree=4095388/4096564
[pid=685] ppid=683 vsize=916400 CPUtime=1767.91
/proc/685/stat : 685 (java) R 683 685 31949 0 -1 0 155875 0 15 0 176689 102 0 0 25 0 8 0 183331160 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4074009649 0 4 0 23759 18446744073709551615 0 0 17 1 0 0
/proc/685/statm: 229100 155580 2546 14 0 215569 0
[pid=685/tid=687] ppid=683 vsize=916400 CPUtime=1.63
/proc/685/task/687/stat : 687 (java) S 683 685 31949 0 -1 64 138700 0 0 0 113 50 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=688] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/688/stat : 688 (java) S 683 685 31949 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=689] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/689/stat : 689 (java) S 683 685 31949 0 -1 64 5 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=690] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/690/stat : 690 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 18 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=691] ppid=683 vsize=916400 CPUtime=0.29
/proc/685/task/691/stat : 691 (java) S 683 685 31949 0 -1 64 505 0 0 0 28 1 0 0 16 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=692] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/692/stat : 692 (java) S 683 685 31949 0 -1 64 0 0 0 0 0 0 0 0 19 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=693] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/693/stat : 693 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 1767.91
Current children cumulated vsize (KiB) 916400



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1816.68 s]
/proc/loadavg: 2.00 2.01 2.44 3/93 773
/proc/meminfo: memFree=1209784/2055888 swapFree=4095388/4096564
[pid=685] ppid=683 vsize=916400 CPUtime=1800.05
/proc/685/stat : 685 (java) R 683 685 31949 0 -1 0 155875 0 15 0 179903 102 0 0 25 0 8 0 183331160 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4074035165 0 4 0 23759 18446744073709551615 0 0 17 1 0 0
/proc/685/statm: 229100 155580 2546 14 0 215569 0
[pid=685/tid=687] ppid=683 vsize=916400 CPUtime=1.63
/proc/685/task/687/stat : 687 (java) S 683 685 31949 0 -1 64 138700 0 0 0 113 50 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=688] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/688/stat : 688 (java) S 683 685 31949 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=689] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/689/stat : 689 (java) S 683 685 31949 0 -1 64 5 0 0 0 0 0 0 0 16 0 8 0 183331898 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=690] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/690/stat : 690 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 18 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=691] ppid=683 vsize=916400 CPUtime=0.29
/proc/685/task/691/stat : 691 (java) S 683 685 31949 0 -1 64 505 0 0 0 28 1 0 0 16 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=685/tid=692] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/692/stat : 692 (java) S 683 685 31949 0 -1 64 0 0 0 0 0 0 0 0 19 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=685/tid=693] ppid=683 vsize=916400 CPUtime=0
/proc/685/task/693/stat : 693 (java) S 683 685 31949 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 183331990 938393600 155580 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 1800.05
Current children cumulated vsize (KiB) 916400

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Child status: 143
Real time (s): 1816.84
CPU time (s): 1800.21
CPU user time (s): 1799.07
CPU system time (s): 1.13383
CPU usage (%): 99.0846
Max. virtual memory (cumulated for all children) (KiB): 916500

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1799.07
system time used= 1.13383
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 155891
page faults= 15
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 38321
involuntary context switches= 59270

runsolver used 1.79673 s user time and 4.69229 s system time

The end

Launcher Data (download as text)

Begin job on node76 on Tue Jan  9 13:29:00 UTC 2007


IDJOB= 237532
IDBENCH= 6947
FILE ID= node76/237532-1168349095

PBS_JOBID= 3503525

Free space on /tmp= 50128 MiB

BENCH NAME= HOME/pub/bench/CPAI06/haystacks/haystacks_26.xml
COMMAND LINE= java -Xms756M -Xmx756M -XX:PermSize=10m -XX:MaxPermSize=10m -jar abscon109.jar esac.xml 1 XSax /tmp/evaluation/237532-1168349095/unknown.xml
RUNSOLVER COMMAND LINE= runsolver  --timestamp  -w ROOT/results/node76/watcher-237532-1168349095 -o ROOT/results/node76/solver-237532-1168349095 -C 1800 -M 900  java -Xms756M -Xmx756M -XX:PermSize=10m -XX:MaxPermSize=10m -jar abscon109.jar esac.xml 1 XSax /tmp/evaluation/237532-1168349095/unknown.xml

META MD5SUM SOLVER= 9b91398c18c9fe54a6c4cfb50dadce08
MD5SUM BENCH=  b92463ea8cbb51f9a3234236cd8f2202

RANDOM SEED= 430574951

TIME LIMIT= 1800 seconds

MEMORY LIMIT= 900 MiB


/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.254
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 6006.17
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.254
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5999.42
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:      2055888 kB
MemFree:         15472 kB
Buffers:          5404 kB
Cached:        1818004 kB
SwapCached:        352 kB
Active:         139860 kB
Inactive:      1803540 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      2055888 kB
LowFree:         15472 kB
SwapTotal:     4096564 kB
SwapFree:      4095388 kB
Dirty:          312364 kB
Writeback:           0 kB
Mapped:         135636 kB
Slab:            82340 kB
Committed_AS:  7086936 kB
PageTables:       2048 kB
VmallocTotal: 536870911 kB
VmallocUsed:    264388 kB
VmallocChunk: 536606243 kB
HugePages_Total:     0
HugePages_Free:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 66645 MiB



End job on node76 on Tue Jan  9 13:59:18 UTC 2007