Trace number 238806

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds, and are wall clock time (not CPU time).

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
sugar 0.40? (TO) 1800.08 1808.2

General information on the benchmark

NameblackHole/BH-4-7/
BlackHole-4-7-e-5_ext.xml
MD5SUM4c58eb69aa8058c4fc80a5c46de30e6e
Bench Category2-ARY-EXT (binary constraints in extension)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables112
Number of constraints1262
Maximum constraint arity2
Maximum domain size28
Number of constraints which are defined in extension1262
Number of constraints which are defined in intension0
Global constraints used (with number of constraints)

Solver Data (download as text)

0.07	c # Sugar v0-40 (/tmp/evaluation/238806-1170108774/sugar-v0-40.jar)
0.07	
0.07	c # MAIN  BEGIN : /tmp/evaluation/238806-1170108774/unknown.xml [Mon Jan 29 23:12:55 2007]
0.07	c # CONVERT_XML  BEGIN : /tmp/evaluation/238806-1170108774/unknown.xml /tmp/evaluation/238806-1170108774/sugar27316.csp [Mon Jan 29 23:12:55 2007]
0.07	c # CMD : java -Xms200M -Xmx600M -jar /tmp/evaluation/238806-1170108774/sugar-v0-40.jar -convert /tmp/evaluation/238806-1170108774/unknown.xml /tmp/evaluation/238806-1170108774/sugar27316.csp
0.28	c BEGIN convert /tmp/evaluation/238806-1170108774/unknown.xml
0.91	c END convert (0.621938 seconds)
0.92	c # CONVERT_XML  END : /tmp/evaluation/238806-1170108774/sugar27316.csp 1866375 bytes (0 seconds) [Mon Jan 29 23:12:55 2007]
0.92	c # CONVERT_XML  CPU : 0.7 = 0 + 0 + 0.62 + 0.08
0.92	
0.92	c # GENERATE_SAT  BEGIN : /tmp/evaluation/238806-1170108774/sugar27316.bcnf [Mon Jan 29 23:12:55 2007]
0.92	c # CMD : java -Xms200M -Xmx600M -jar /tmp/evaluation/238806-1170108774/sugar-v0-40.jar -bcnf -map /tmp/evaluation/238806-1170108774/sugar27316.map /tmp/evaluation/238806-1170108774/sugar27316.csp /tmp/evaluation/238806-1170108774/sugar27316.bcnf
1.03	c BEGIN parse /tmp/evaluation/238806-1170108774/sugar27316.csp
114.33	c END parse (113.303871 seconds)
114.33	c BEGIN simplify
114.77	c END simplify (0.432883 seconds)
114.77	c BEGIN encode /tmp/evaluation/238806-1170108774/sugar27316.bcnf
117.51	c END encode (2.73969 seconds)
117.78	c # GENERATE_SAT  END : 18424 variables 91071 clauses 26422124 bytes (117 seconds) [Mon Jan 29 23:14:52 2007]
117.78	c # GENERATE_SAT  CPU : 0.15 = 0.14 + 0.01 + 0 + 0
117.78	
117.78	c # SOLVE_SAT  BEGIN : /tmp/evaluation/238806-1170108774/sugar27316.bcnf [Mon Jan 29 23:14:52 2007]
117.78	c # CMD : /tmp/evaluation/238806-1170108774/minisat /tmp/evaluation/238806-1170108774/sugar27316.bcnf /tmp/evaluation/238806-1170108774/sugar27316.out
118.71	c ==================================[MINISAT]===================================
118.71	c | Conflicts |     ORIGINAL     |              LEARNT              | Progress |
118.71	c |           | Clauses Literals |   Limit Clauses Literals  Lit/Cl |          |
118.71	c ==============================================================================
118.71	c |         0 |   88578  6318492 |   29526       0        0     nan |  0.000 % |
118.77	c |       100 |   88584  6318492 |   32478      94    17653   187.8 |  0.456 % |
118.81	c |       250 |   88598  6318492 |   35726     229    18815    82.2 |  0.651 % |
118.85	c |       475 |   88608  6318492 |   39299     444    21080    47.5 |  0.651 % |
118.91	c |       812 |   88620  6318492 |   43229     769    24505    31.9 |  0.651 % |
118.98	c |      1319 |   88634  6318492 |   47551    1262    30472    24.1 |  0.651 % |
119.10	c |      2078 |   88657  6318492 |   52307    1993    39288    19.7 |  0.760 % |
119.27	c |      3218 |   88676  6318492 |   57537    3113    57807    18.6 |  0.798 % |
119.47	c |      4927 |   88685  6318492 |   63291    4813   118913    24.7 |  0.798 % |
119.76	c |      7489 |   88690  6318492 |   69620    7370   214549    29.1 |  0.798 % |
120.41	c |     11334 |   88694  6318492 |   76582   11211   404304    36.1 |  0.798 % |
121.40	c |     17100 |   88694  6318492 |   84241   16977   761058    44.8 |  0.798 % |
123.05	c |     25751 |   88705  6318492 |   92665   25617  1229845    48.0 |  0.798 % |
127.28	c |     38725 |   88711  6318492 |  101931   38585  2228878    57.8 |  0.798 % |
134.95	c |     58186 |   88714  6318492 |  112124   58043  3890902    67.0 |  0.798 % |
150.37	c |     87378 |   88573  6314437 |  123337   87217  6252547    71.7 |  0.798 % |
176.11	c |    131169 |   88581  6314437 |  135671   20837  1666892    80.0 |  0.798 % |
204.28	c |    196853 |   88614  6314437 |  149238   86488  5714527    66.1 |  0.798 % |
265.07	c |    295379 |   88633  6314437 |  164162   54134  3875952    71.6 |  0.798 % |
358.10	c |    443168 |   88644  6314437 |  180578   56939  3380018    59.4 |  0.798 % |
506.44	c |    664853 |   88646  6314420 |  198636  118523  5980175    50.5 |  0.836 % |
809.65	c |    997378 |   88667  6314420 |  218499  102340  3942678    38.5 |  0.836 % |
1335.09	c |   1496167 |   88676  6314300 |  240349  215223 12964328    60.2 |  0.847 % |
1806.13	
1806.13	s UNKNOWN
1806.13	c # INTERRUPTED
1806.13	c # MAIN  END :  (1806 seconds) [Mon Jan 29 23:43:01 2007]
1806.13	c # MAIN  CPU : 0.86 = 0.15 + 0.01 + 0.62 + 0.08
1806.13

Verifier Data (download as text)

ERROR: Unexpected answer ! (SAT/UNSAT expected)
Got answer: UNKNOWN

Watcher Data (download as text)

runsolver version 3.1.3 (c) roussel@cril.univ-artois.fr

command line: runsolver --timestamp -w ROOT/results/node7/watcher-238806-1170108774 -o ROOT/results/node7/solver-238806-1170108774 -C 1800 -M 900 /tmp/evaluation/238806-1170108774/solver /tmp/evaluation/238806-1170108774 /tmp/evaluation/238806-1170108774/unknown.xml 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 921600 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 972800 KiB
Current StackSize limit: 10240 KiB

/proc/loadavg: 3.49 3.79 3.30 5/94 27317
/proc/meminfo: memFree=65336/2055920 swapFree=4192812/4192956
[pid=27316] ppid=27314 vsize=18540 CPUtime=0
/proc/27316/stat : 27316 (runsolver) R 27314 27316 27251 0 -1 4194368 16 0 0 0 0 0 0 0 20 0 1 0 30832170 18984960 279 18446744073709551615 4194304 4267372 548682069040 18446744073709551615 244671114535 0 0 4096 24578 0 0 0 17 1 0 0
/proc/27316/statm: 4635 279 244 17 0 2626 0

[startup+0.108621 s]
/proc/loadavg: 3.49 3.79 3.30 5/94 27317
/proc/meminfo: memFree=65336/2055920 swapFree=4192812/4192956
[pid=27316] ppid=27314 vsize=13724 CPUtime=0.04
/proc/27316/stat : 27316 (sugar) S 27314 27316 27251 0 -1 4194304 1112 0 0 0 3 1 0 0 19 0 1 0 30832170 14053376 805 18446744073709551615 4194304 4206940 548682069232 18446744073709551615 244680011874 0 0 4224 16386 18446744071563648864 0 0 17 1 0 0
/proc/27316/statm: 3431 805 368 3 0 531 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 13724

[startup+0.516667 s]
/proc/loadavg: 3.49 3.79 3.30 5/94 27317
/proc/meminfo: memFree=65336/2055920 swapFree=4192812/4192956
[pid=27316] ppid=27314 vsize=13724 CPUtime=0.04
/proc/27316/stat : 27316 (sugar) S 27314 27316 27251 0 -1 4194304 1121 0 0 0 3 1 0 0 15 0 1 0 30832170 14053376 805 18446744073709551615 4194304 4206940 548682069232 18446744073709551615 244680011874 0 0 4224 16386 18446744071563648864 0 0 17 1 0 0
/proc/27316/statm: 3431 805 368 3 0 531 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 13724

[startup+1.33975 s]
/proc/loadavg: 3.49 3.79 3.30 3/102 27333
/proc/meminfo: memFree=27960/2055920 swapFree=4192812/4192956
[pid=27316] ppid=27314 vsize=13724 CPUtime=0.74
/proc/27316/stat : 27316 (sugar) S 27314 27316 27251 0 -1 4194304 1190 7306 0 33 3 1 62 8 15 0 1 0 30832170 14053376 809 18446744073709551615 4194304 4206940 548682069232 18446744073709551615 244680011874 0 0 4224 16386 18446744071563648864 0 0 17 0 0 0
/proc/27316/statm: 3431 809 371 3 0 531 0
[pid=27326] ppid=27316 vsize=807064 CPUtime=0.39
/proc/27326/stat : 27326 (java) R 27316 27316 27251 0 -1 0 7254 0 1 0 36 3 0 0 25 0 8 0 30832262 826433536 7003 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/27326/statm: 201766 7003 1494 14 0 189044 0
[pid=27326/tid=27327] ppid=27316 vsize=807064 CPUtime=0.05
/proc/27326/task/27327/stat : 27327 (java) R 27316 27316 27251 0 -1 64 1202 0 0 0 5 0 0 0 16 0 8 0 30832265 826433536 7003 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4153968433 0 0 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27328] ppid=27316 vsize=807064 CPUtime=0
/proc/27326/task/27328/stat : 27328 (java) S 27316 27316 27251 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 30832266 826433536 7003 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27329] ppid=27316 vsize=807064 CPUtime=0
/proc/27326/task/27329/stat : 27329 (java) S 27316 27316 27251 0 -1 64 5 0 0 0 0 0 0 0 15 0 8 0 30832266 826433536 7003 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27330] ppid=27316 vsize=807064 CPUtime=0
/proc/27326/task/27330/stat : 27330 (java) S 27316 27316 27251 0 -1 64 1 0 0 0 0 0 0 0 23 0 8 0 30832269 826433536 7004 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27331] ppid=27316 vsize=807064 CPUtime=0.05
/proc/27326/task/27331/stat : 27331 (java) S 27316 27316 27251 0 -1 64 292 0 0 0 5 0 0 0 17 0 8 0 30832269 826433536 7005 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27332] ppid=27316 vsize=807064 CPUtime=0
/proc/27326/task/27332/stat : 27332 (java) S 27316 27316 27251 0 -1 64 3 0 0 0 0 0 0 0 24 0 8 0 30832269 826433536 7006 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27333] ppid=27316 vsize=807064 CPUtime=0
/proc/27326/task/27333/stat : 27333 (java) S 27316 27316 27251 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 30832269 826433536 7007 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 1.13
Current children cumulated vsize (KiB) 820788

[startup+2.99393 s]
/proc/loadavg: 3.49 3.79 3.30 4/102 27333
/proc/meminfo: memFree=15224/2055920 swapFree=4192812/4192956
[pid=27316] ppid=27314 vsize=13724 CPUtime=0.74
/proc/27316/stat : 27316 (sugar) S 27314 27316 27251 0 -1 4194304 1190 7306 0 33 3 1 62 8 15 0 1 0 30832170 14053376 809 18446744073709551615 4194304 4206940 548682069232 18446744073709551615 244680011874 0 0 4224 16386 18446744071563648864 0 0 17 0 0 0
/proc/27316/statm: 3431 809 371 3 0 531 0
[pid=27326] ppid=27316 vsize=807064 CPUtime=2.03
/proc/27326/stat : 27326 (java) R 27316 27316 27251 0 -1 0 14285 0 1 0 196 7 0 0 25 0 8 0 30832262 826433536 14018 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4073481404 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/27326/statm: 201766 14018 1494 14 0 189044 0
[pid=27326/tid=27327] ppid=27316 vsize=807064 CPUtime=0.55
/proc/27326/task/27327/stat : 27327 (java) S 27316 27316 27251 0 -1 64 8217 0 0 0 52 3 0 0 16 0 8 0 30832265 826433536 14018 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27328] ppid=27316 vsize=807064 CPUtime=0
/proc/27326/task/27328/stat : 27328 (java) S 27316 27316 27251 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 30832266 826433536 14018 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27329] ppid=27316 vsize=807064 CPUtime=0
/proc/27326/task/27329/stat : 27329 (java) S 27316 27316 27251 0 -1 64 5 0 0 0 0 0 0 0 15 0 8 0 30832266 826433536 14018 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27330] ppid=27316 vsize=807064 CPUtime=0
/proc/27326/task/27330/stat : 27330 (java) S 27316 27316 27251 0 -1 64 1 0 0 0 0 0 0 0 23 0 8 0 30832269 826433536 14018 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27331] ppid=27316 vsize=807064 CPUtime=0.05
/proc/27326/task/27331/stat : 27331 (java) S 27316 27316 27251 0 -1 64 292 0 0 0 5 0 0 0 16 0 8 0 30832269 826433536 14018 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27332] ppid=27316 vsize=807064 CPUtime=0
/proc/27326/task/27332/stat : 27332 (java) S 27316 27316 27251 0 -1 64 3 0 0 0 0 0 0 0 24 0 8 0 30832269 826433536 14018 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27333] ppid=27316 vsize=807064 CPUtime=0
/proc/27326/task/27333/stat : 27333 (java) S 27316 27316 27251 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 30832269 826433536 14018 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 2.77
Current children cumulated vsize (KiB) 820788

[startup+6.19626 s]
/proc/loadavg: 3.53 3.79 3.30 3/102 27333
/proc/meminfo: memFree=15416/2055920 swapFree=4192812/4192956
[pid=27316] ppid=27314 vsize=13724 CPUtime=0.74
/proc/27316/stat : 27316 (sugar) S 27314 27316 27251 0 -1 4194304 1190 7306 0 33 3 1 62 8 15 0 1 0 30832170 14053376 809 18446744073709551615 4194304 4206940 548682069232 18446744073709551615 244680011874 0 0 4224 16386 18446744071563648864 0 0 17 0 0 0
/proc/27316/statm: 3431 809 371 3 0 531 0
[pid=27326] ppid=27316 vsize=807332 CPUtime=5.16
/proc/27326/stat : 27326 (java) R 27316 27316 27251 0 -1 0 41777 0 1 0 493 23 0 0 25 0 8 0 30832262 826707968 41391 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4153380222 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/27326/statm: 201833 41391 1506 14 0 189111 0
[pid=27326/tid=27327] ppid=27316 vsize=807332 CPUtime=1.66
/proc/27326/task/27327/stat : 27327 (java) S 27316 27316 27251 0 -1 64 35429 0 0 0 151 15 0 0 15 0 8 0 30832265 826707968 41391 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27328] ppid=27316 vsize=807332 CPUtime=0
/proc/27326/task/27328/stat : 27328 (java) S 27316 27316 27251 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 30832266 826707968 41391 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27329] ppid=27316 vsize=807332 CPUtime=0
/proc/27326/task/27329/stat : 27329 (java) S 27316 27316 27251 0 -1 64 5 0 0 0 0 0 0 0 15 0 8 0 30832266 826707968 41391 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27330] ppid=27316 vsize=807332 CPUtime=0
/proc/27326/task/27330/stat : 27330 (java) S 27316 27316 27251 0 -1 64 1 0 0 0 0 0 0 0 23 0 8 0 30832269 826707968 41391 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27331] ppid=27316 vsize=807332 CPUtime=0.12
/proc/27326/task/27331/stat : 27331 (java) S 27316 27316 27251 0 -1 64 400 0 0 0 12 0 0 0 16 0 8 0 30832269 826707968 41391 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27332] ppid=27316 vsize=807332 CPUtime=0
/proc/27326/task/27332/stat : 27332 (java) S 27316 27316 27251 0 -1 64 3 0 0 0 0 0 0 0 24 0 8 0 30832269 826707968 41391 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27333] ppid=27316 vsize=807332 CPUtime=0
/proc/27326/task/27333/stat : 27333 (java) S 27316 27316 27251 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 30832269 826707968 41391 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 5.9
Current children cumulated vsize (KiB) 821056

[startup+12.6899 s]
/proc/loadavg: 3.49 3.78 3.30 3/102 27333
/proc/meminfo: memFree=15928/2055920 swapFree=4192812/4192956
[pid=27316] ppid=27314 vsize=13724 CPUtime=0.74
/proc/27316/stat : 27316 (sugar) S 27314 27316 27251 0 -1 4194304 1190 7306 0 33 3 1 62 8 15 0 1 0 30832170 14053376 809 18446744073709551615 4194304 4206940 548682069232 18446744073709551615 244680011874 0 0 4224 16386 18446744071563648864 0 0 17 0 0 0
/proc/27316/statm: 3431 809 371 3 0 531 0
[pid=27326] ppid=27316 vsize=807332 CPUtime=11.59
/proc/27326/stat : 27326 (java) R 27316 27316 27251 0 -1 0 55125 0 1 0 1117 42 0 0 25 0 8 0 30832262 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4073746962 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/27326/statm: 201833 54123 1507 14 0 189111 0
[pid=27326/tid=27327] ppid=27316 vsize=807332 CPUtime=4.33
/proc/27326/task/27327/stat : 27327 (java) S 27316 27316 27251 0 -1 64 48507 0 0 0 406 27 0 0 16 0 8 0 30832265 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27328] ppid=27316 vsize=807332 CPUtime=0
/proc/27326/task/27328/stat : 27328 (java) S 27316 27316 27251 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 30832266 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27329] ppid=27316 vsize=807332 CPUtime=0
/proc/27326/task/27329/stat : 27329 (java) S 27316 27316 27251 0 -1 64 5 0 0 0 0 0 0 0 15 0 8 0 30832266 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27330] ppid=27316 vsize=807332 CPUtime=0
/proc/27326/task/27330/stat : 27330 (java) S 27316 27316 27251 0 -1 64 1 0 0 0 0 0 0 0 23 0 8 0 30832269 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27331] ppid=27316 vsize=807332 CPUtime=0.13
/proc/27326/task/27331/stat : 27331 (java) S 27316 27316 27251 0 -1 64 403 0 0 0 13 0 0 0 16 0 8 0 30832269 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27332] ppid=27316 vsize=807332 CPUtime=0
/proc/27326/task/27332/stat : 27332 (java) S 27316 27316 27251 0 -1 64 3 0 0 0 0 0 0 0 24 0 8 0 30832269 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27333] ppid=27316 vsize=807332 CPUtime=0
/proc/27326/task/27333/stat : 27333 (java) S 27316 27316 27251 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 30832269 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 12.33
Current children cumulated vsize (KiB) 821056

[startup+25.5553 s]
/proc/loadavg: 3.37 3.74 3.29 3/102 27333
/proc/meminfo: memFree=15096/2055920 swapFree=4192812/4192956
[pid=27316] ppid=27314 vsize=13724 CPUtime=0.74
/proc/27316/stat : 27316 (sugar) S 27314 27316 27251 0 -1 4194304 1190 7306 0 33 3 1 62 8 15 0 1 0 30832170 14053376 809 18446744073709551615 4194304 4206940 548682069232 18446744073709551615 244680011874 0 0 4224 16386 18446744071563648864 0 0 17 0 0 0
/proc/27316/statm: 3431 809 371 3 0 531 0
[pid=27326] ppid=27316 vsize=807332 CPUtime=24.32
/proc/27326/stat : 27326 (java) R 27316 27316 27251 0 -1 0 56220 0 1 0 2376 56 0 0 25 0 8 0 30832262 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4073332706 0 4 0 23759 18446744073709551615 0 0 17 0 0 0
/proc/27326/statm: 201833 54123 1507 14 0 189111 0
[pid=27326/tid=27327] ppid=27316 vsize=807332 CPUtime=9.75
/proc/27326/task/27327/stat : 27327 (java) S 27316 27316 27251 0 -1 64 49093 0 0 0 943 32 0 0 16 0 8 0 30832265 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27328] ppid=27316 vsize=807332 CPUtime=0
/proc/27326/task/27328/stat : 27328 (java) S 27316 27316 27251 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 30832266 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27329] ppid=27316 vsize=807332 CPUtime=0
/proc/27326/task/27329/stat : 27329 (java) S 27316 27316 27251 0 -1 64 5 0 0 0 0 0 0 0 15 0 8 0 30832266 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27330] ppid=27316 vsize=807332 CPUtime=0
/proc/27326/task/27330/stat : 27330 (java) S 27316 27316 27251 0 -1 64 1 0 0 0 0 0 0 0 23 0 8 0 30832269 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27331] ppid=27316 vsize=807332 CPUtime=0.13
/proc/27326/task/27331/stat : 27331 (java) S 27316 27316 27251 0 -1 64 403 0 0 0 13 0 0 0 16 0 8 0 30832269 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27332] ppid=27316 vsize=807332 CPUtime=0
/proc/27326/task/27332/stat : 27332 (java) S 27316 27316 27251 0 -1 64 3 0 0 0 0 0 0 0 24 0 8 0 30832269 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27333] ppid=27316 vsize=807332 CPUtime=0
/proc/27326/task/27333/stat : 27333 (java) S 27316 27316 27251 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 30832269 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 25.06
Current children cumulated vsize (KiB) 821056

[startup+51.1708 s]
/proc/loadavg: 3.11 3.65 3.27 3/102 27333
/proc/meminfo: memFree=15928/2055920 swapFree=4192812/4192956
[pid=27316] ppid=27314 vsize=13724 CPUtime=0.74
/proc/27316/stat : 27316 (sugar) S 27314 27316 27251 0 -1 4194304 1190 7306 0 33 3 1 62 8 15 0 1 0 30832170 14053376 809 18446744073709551615 4194304 4206940 548682069232 18446744073709551615 244680011874 0 0 4224 16386 18446744071563648864 0 0 17 0 0 0
/proc/27316/statm: 3431 809 371 3 0 531 0
[pid=27326] ppid=27316 vsize=807332 CPUtime=49.69
/proc/27326/stat : 27326 (java) R 27316 27316 27251 0 -1 0 59277 0 1 0 4892 77 0 0 25 0 8 0 30832262 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 18446744073709551615 0 0 17 1 0 0
/proc/27326/statm: 201833 54123 1507 14 0 189111 0
[pid=27326/tid=27327] ppid=27316 vsize=807332 CPUtime=21.06
/proc/27326/task/27327/stat : 27327 (java) S 27316 27316 27251 0 -1 64 51213 0 0 0 2063 43 0 0 15 0 8 0 30832265 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=27326/tid=27328] ppid=27316 vsize=807332 CPUtime=0
/proc/27326/task/27328/stat : 27328 (java) S 27316 27316 27251 0 -1 64 11 0 0 0 0 0 0 0 16 0 8 0 30832266 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27329] ppid=27316 vsize=807332 CPUtime=0
/proc/27326/task/27329/stat : 27329 (java) S 27316 27316 27251 0 -1 64 5 0 0 0 0 0 0 0 15 0 8 0 30832266 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27330] ppid=27316 vsize=807332 CPUtime=0
/proc/27326/task/27330/stat : 27330 (java) S 27316 27316 27251 0 -1 64 1 0 0 0 0 0 0 0 23 0 8 0 30832269 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27331] ppid=27316 vsize=807332 CPUtime=0.13
/proc/27326/task/27331/stat : 27331 (java) S 27316 27316 27251 0 -1 64 403 0 0 0 13 0 0 0 17 0 8 0 30832269 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27332] ppid=27316 vsize=807332 CPUtime=0
/proc/27326/task/27332/stat : 27332 (java) S 27316 27316 27251 0 -1 64 3 0 0 0 0 0 0 0 24 0 8 0 30832269 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 0 0 0
[pid=27326/tid=27333] ppid=27316 vsize=807332 CPUtime=0
/proc/27326/task/27333/stat : 27333 (java) S 27316 27316 27251 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 30832269 826707968 54123 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 50.43
Current children cumulated vsize (KiB) 821056

[startup+102.424 s]
/proc/loadavg: 2.92 3.50 3.24 3/102 27333
/proc/meminfo: memFree=15352/2055920 swapFree=4192812/4192956
[pid=27316] ppid=27314 vsize=13724 CPUtime=0.74
/proc/27316/stat : 27316 (sugar) S 27314 27316 27251 0 -1 4194304 1190 7306 0 33 3 1 62 8 15 0 1 0 30832170 14053376 809 18446744073709551615 4194304 4206940 548682069232 18446744073709551615 244680011874 0 0 4224 16386 18446744071563648864 0 0 17 0 0 0
/proc/27316/statm: 3431 809 371 3 0 531 0
[pid=27326] ppid=27316 vsize=807332 CPUtime=100.4
/proc/27326/stat : 27326 (java) R 27316 27316 27251 0 -1 0 90987 0 1 0 9909 131 0 0 25 0 8 0 30832262 826707968 81331 18446744073709551615 134512640 134570532 4294956480 18446744073709551615 4294960144 0 4 0 23759 18446744073709551615 0 0 17 1 0 0
/proc/27326/statm: 201833 81331 1507 14 0 189111 0
[pid=27326/tid=27327] ppid=27316 vsize=807332 CPUtime=44.27

################
# More data... #
################

Current children cumulated CPU time (s) 1238.38
Current children cumulated vsize (KiB) 121736

[startup+1303.3 s]
/proc/loadavg: 2.91 2.83 2.67 3/96 28886
/proc/meminfo: memFree=16312/2055920 swapFree=4192812/4192956
[pid=27316] ppid=27314 vsize=23800 CPUtime=0.9
/proc/27316/stat : 27316 (sugar) S 27314 27316 27251 0 -1 4194304 3691 7306 0 33 18 2 62 8 16 0 1 0 30832170 24371200 3170 18446744073709551615 4194304 4206940 548682069232 18446744073709551615 244680011874 0 0 4224 16386 18446744071563648864 0 0 17 1 0 0
/proc/27316/statm: 5950 3170 373 3 0 3050 0
[pid=27326] ppid=27316 vsize=0 CPUtime=115.21
/proc/27326/stat : 27326 (java) Z 27316 27316 27251 0 -1 1036 101670 0 2 0 11358 163 0 0 16 0 1 0 30832262 0 0 18446744073709551615 0 0 0 0 0 0 4 0 23759 18446744071563351923 0 0 17 1 0 0
/proc/27326/statm: 0 0 0 0 0 0 0
[pid=27335] ppid=27316 vsize=98100 CPUtime=1182.16
/proc/27335/stat : 27335 (minisat) R 27316 27316 27251 0 -1 0 25036 0 5 0 117923 293 0 0 25 0 1 0 30843949 100454400 24002 18446744073709551615 134512640 134974460 4294956768 18446744073709551615 134528363 0 0 4096 3 0 0 0 17 1 0 0
/proc/27335/statm: 24525 24002 52 112 0 24409 0
Current children cumulated CPU time (s) 1298.27
Current children cumulated vsize (KiB) 121900

[startup+1363.31 s]
/proc/loadavg: 3.91 3.09 2.76 3/96 28888
/proc/meminfo: memFree=14904/2055920 swapFree=4192812/4192956
[pid=27316] ppid=27314 vsize=23800 CPUtime=0.9
/proc/27316/stat : 27316 (sugar) S 27314 27316 27251 0 -1 4194304 3691 7306 0 33 18 2 62 8 16 0 1 0 30832170 24371200 3170 18446744073709551615 4194304 4206940 548682069232 18446744073709551615 244680011874 0 0 4224 16386 18446744071563648864 0 0 17 1 0 0
/proc/27316/statm: 5950 3170 373 3 0 3050 0
[pid=27326] ppid=27316 vsize=0 CPUtime=115.21
/proc/27326/stat : 27326 (java) Z 27316 27316 27251 0 -1 1036 101670 0 2 0 11358 163 0 0 16 0 1 0 30832262 0 0 18446744073709551615 0 0 0 0 0 0 4 0 23759 18446744071563351923 0 0 17 1 0 0
/proc/27326/statm: 0 0 0 0 0 0 0
[pid=27335] ppid=27316 vsize=103452 CPUtime=1242.05
/proc/27335/stat : 27335 (minisat) R 27316 27316 27251 0 -1 0 26334 0 5 0 123885 320 0 0 25 0 1 0 30843949 105934848 25300 18446744073709551615 134512640 134974460 4294956768 18446744073709551615 134528363 0 0 4096 3 0 0 0 17 1 0 0
/proc/27335/statm: 25863 25300 52 112 0 25747 0
Current children cumulated CPU time (s) 1358.16
Current children cumulated vsize (KiB) 127252

[startup+1423.37 s]
/proc/loadavg: 3.49 3.15 2.81 4/96 28888
/proc/meminfo: memFree=16760/2055920 swapFree=4192812/4192956
[pid=27316] ppid=27314 vsize=23800 CPUtime=0.9
/proc/27316/stat : 27316 (sugar) S 27314 27316 27251 0 -1 4194304 3691 7306 0 33 18 2 62 8 16 0 1 0 30832170 24371200 3170 18446744073709551615 4194304 4206940 548682069232 18446744073709551615 244680011874 0 0 4224 16386 18446744071563648864 0 0 17 1 0 0
/proc/27316/statm: 5950 3170 373 3 0 3050 0
[pid=27326] ppid=27316 vsize=0 CPUtime=115.21
/proc/27326/stat : 27326 (java) Z 27316 27316 27251 0 -1 1036 101670 0 2 0 11358 163 0 0 16 0 1 0 30832262 0 0 18446744073709551615 0 0 0 0 0 0 4 0 23759 18446744071563351923 0 0 17 1 0 0
/proc/27326/statm: 0 0 0 0 0 0 0
[pid=27335] ppid=27316 vsize=104624 CPUtime=1301.96
/proc/27335/stat : 27335 (minisat) R 27316 27316 27251 0 -1 0 26652 0 5 0 129848 348 0 0 25 0 1 0 30843949 107134976 25618 18446744073709551615 134512640 134974460 4294956768 18446744073709551615 134528535 0 0 4096 3 0 0 0 17 1 0 0
/proc/27335/statm: 26156 25618 52 112 0 26040 0
Current children cumulated CPU time (s) 1418.07
Current children cumulated vsize (KiB) 128424

[startup+1483.42 s]
/proc/loadavg: 2.86 3.03 2.79 3/96 28888
/proc/meminfo: memFree=15032/2055920 swapFree=4192812/4192956
[pid=27316] ppid=27314 vsize=23800 CPUtime=0.9
/proc/27316/stat : 27316 (sugar) S 27314 27316 27251 0 -1 4194304 3691 7306 0 33 18 2 62 8 16 0 1 0 30832170 24371200 3170 18446744073709551615 4194304 4206940 548682069232 18446744073709551615 244680011874 0 0 4224 16386 18446744071563648864 0 0 17 1 0 0
/proc/27316/statm: 5950 3170 373 3 0 3050 0
[pid=27326] ppid=27316 vsize=0 CPUtime=115.21
/proc/27326/stat : 27326 (java) Z 27316 27316 27251 0 -1 1036 101670 0 2 0 11358 163 0 0 16 0 1 0 30832262 0 0 18446744073709551615 0 0 0 0 0 0 4 0 23759 18446744071563351923 0 0 17 1 0 0
/proc/27326/statm: 0 0 0 0 0 0 0
[pid=27335] ppid=27316 vsize=104800 CPUtime=1361.91
/proc/27335/stat : 27335 (minisat) R 27316 27316 27251 0 -1 0 26704 0 5 0 135818 373 0 0 25 0 1 0 30843949 107315200 25670 18446744073709551615 134512640 134974460 4294956768 18446744073709551615 134528363 0 0 4096 3 0 0 0 17 1 0 0
/proc/27335/statm: 26200 25670 52 112 0 26084 0
Current children cumulated CPU time (s) 1478.02
Current children cumulated vsize (KiB) 128600

[startup+1543.45 s]
/proc/loadavg: 2.58 2.92 2.76 3/96 28888
/proc/meminfo: memFree=15864/2055920 swapFree=4192812/4192956
[pid=27316] ppid=27314 vsize=23800 CPUtime=0.9
/proc/27316/stat : 27316 (sugar) S 27314 27316 27251 0 -1 4194304 3691 7306 0 33 18 2 62 8 16 0 1 0 30832170 24371200 3170 18446744073709551615 4194304 4206940 548682069232 18446744073709551615 244680011874 0 0 4224 16386 18446744071563648864 0 0 17 1 0 0
/proc/27316/statm: 5950 3170 373 3 0 3050 0
[pid=27326] ppid=27316 vsize=0 CPUtime=115.21
/proc/27326/stat : 27326 (java) Z 27316 27316 27251 0 -1 1036 101670 0 2 0 11358 163 0 0 16 0 1 0 30832262 0 0 18446744073709551615 0 0 0 0 0 0 4 0 23759 18446744071563351923 0 0 17 1 0 0
/proc/27326/statm: 0 0 0 0 0 0 0
[pid=27335] ppid=27316 vsize=104800 CPUtime=1421.8
/proc/27335/stat : 27335 (minisat) R 27316 27316 27251 0 -1 0 26717 0 5 0 141784 396 0 0 25 0 1 0 30843949 107315200 25683 18446744073709551615 134512640 134974460 4294956768 18446744073709551615 134528363 0 0 4096 3 0 0 0 17 1 0 0
/proc/27335/statm: 26200 25683 52 112 0 26084 0
Current children cumulated CPU time (s) 1537.91
Current children cumulated vsize (KiB) 128600

[startup+1603.53 s]
/proc/loadavg: 2.79 2.90 2.76 3/96 28888
/proc/meminfo: memFree=15608/2055920 swapFree=4192812/4192956
[pid=27316] ppid=27314 vsize=23800 CPUtime=0.9
/proc/27316/stat : 27316 (sugar) S 27314 27316 27251 0 -1 4194304 3691 7306 0 33 18 2 62 8 16 0 1 0 30832170 24371200 3170 18446744073709551615 4194304 4206940 548682069232 18446744073709551615 244680011874 0 0 4224 16386 18446744071563648864 0 0 17 1 0 0
/proc/27316/statm: 5950 3170 373 3 0 3050 0
[pid=27326] ppid=27316 vsize=0 CPUtime=115.21
/proc/27326/stat : 27326 (java) Z 27316 27316 27251 0 -1 1036 101670 0 2 0 11358 163 0 0 16 0 1 0 30832262 0 0 18446744073709551615 0 0 0 0 0 0 4 0 23759 18446744071563351923 0 0 17 1 0 0
/proc/27326/statm: 0 0 0 0 0 0 0
[pid=27335] ppid=27316 vsize=105044 CPUtime=1481.76
/proc/27335/stat : 27335 (minisat) R 27316 27316 27251 0 -1 0 26754 0 5 0 147755 421 0 0 25 0 1 0 30843949 107565056 25720 18446744073709551615 134512640 134974460 4294956768 18446744073709551615 134528677 0 0 4096 3 0 0 0 17 1 0 0
/proc/27335/statm: 26261 25720 52 112 0 26145 0
Current children cumulated CPU time (s) 1597.87
Current children cumulated vsize (KiB) 128844

[startup+1663.59 s]
/proc/loadavg: 3.42 3.04 2.82 3/96 28888
/proc/meminfo: memFree=15864/2055920 swapFree=4192812/4192956
[pid=27316] ppid=27314 vsize=23800 CPUtime=0.9
/proc/27316/stat : 27316 (sugar) S 27314 27316 27251 0 -1 4194304 3691 7306 0 33 18 2 62 8 16 0 1 0 30832170 24371200 3170 18446744073709551615 4194304 4206940 548682069232 18446744073709551615 244680011874 0 0 4224 16386 18446744071563648864 0 0 17 1 0 0
/proc/27316/statm: 5950 3170 373 3 0 3050 0
[pid=27326] ppid=27316 vsize=0 CPUtime=115.21
/proc/27326/stat : 27326 (java) Z 27316 27316 27251 0 -1 1036 101670 0 2 0 11358 163 0 0 16 0 1 0 30832262 0 0 18446744073709551615 0 0 0 0 0 0 4 0 23759 18446744071563351923 0 0 17 1 0 0
/proc/27326/statm: 0 0 0 0 0 0 0
[pid=27335] ppid=27316 vsize=105256 CPUtime=1541.69
/proc/27335/stat : 27335 (minisat) R 27316 27316 27251 0 -1 0 26775 0 5 0 153728 441 0 0 25 0 1 0 30843949 107782144 25741 18446744073709551615 134512640 134974460 4294956768 18446744073709551615 134518928 0 0 4096 3 0 0 0 17 1 0 0
/proc/27335/statm: 26314 25741 52 112 0 26198 0
Current children cumulated CPU time (s) 1657.8
Current children cumulated vsize (KiB) 129056

[startup+1723.68 s]
/proc/loadavg: 4.10 3.31 2.92 3/96 28888
/proc/meminfo: memFree=16184/2055920 swapFree=4192812/4192956
[pid=27316] ppid=27314 vsize=23800 CPUtime=0.9
/proc/27316/stat : 27316 (sugar) S 27314 27316 27251 0 -1 4194304 3691 7306 0 33 18 2 62 8 16 0 1 0 30832170 24371200 3170 18446744073709551615 4194304 4206940 548682069232 18446744073709551615 244680011874 0 0 4224 16386 18446744071563648864 0 0 17 1 0 0
/proc/27316/statm: 5950 3170 373 3 0 3050 0
[pid=27326] ppid=27316 vsize=0 CPUtime=115.21
/proc/27326/stat : 27326 (java) Z 27316 27316 27251 0 -1 1036 101670 0 2 0 11358 163 0 0 16 0 1 0 30832262 0 0 18446744073709551615 0 0 0 0 0 0 4 0 23759 18446744071563351923 0 0 17 1 0 0
/proc/27326/statm: 0 0 0 0 0 0 0
[pid=27335] ppid=27316 vsize=105256 CPUtime=1601.66
/proc/27335/stat : 27335 (minisat) R 27316 27316 27251 0 -1 0 26779 0 5 0 159700 466 0 0 25 0 1 0 30843949 107782144 25745 18446744073709551615 134512640 134974460 4294956768 18446744073709551615 134528363 0 0 4096 3 0 0 0 17 1 0 0
/proc/27335/statm: 26314 25745 52 112 0 26198 0
Current children cumulated CPU time (s) 1717.77
Current children cumulated vsize (KiB) 129056

[startup+1783.7 s]
/proc/loadavg: 3.11 3.17 2.90 3/96 28888
/proc/meminfo: memFree=15480/2055920 swapFree=4192812/4192956
[pid=27316] ppid=27314 vsize=23800 CPUtime=0.9
/proc/27316/stat : 27316 (sugar) S 27314 27316 27251 0 -1 4194304 3691 7306 0 33 18 2 62 8 16 0 1 0 30832170 24371200 3170 18446744073709551615 4194304 4206940 548682069232 18446744073709551615 244680011874 0 0 4224 16386 18446744071563648864 0 0 17 1 0 0
/proc/27316/statm: 5950 3170 373 3 0 3050 0
[pid=27326] ppid=27316 vsize=0 CPUtime=115.21
/proc/27326/stat : 27326 (java) Z 27316 27316 27251 0 -1 1036 101670 0 2 0 11358 163 0 0 16 0 1 0 30832262 0 0 18446744073709551615 0 0 0 0 0 0 4 0 23759 18446744071563351923 0 0 17 1 0 0
/proc/27326/statm: 0 0 0 0 0 0 0
[pid=27335] ppid=27316 vsize=105256 CPUtime=1661.57
/proc/27335/stat : 27335 (minisat) R 27316 27316 27251 0 -1 0 26779 0 5 0 165664 493 0 0 25 0 1 0 30843949 107782144 25745 18446744073709551615 134512640 134974460 4294956768 18446744073709551615 134528723 0 0 4096 3 0 0 0 17 1 0 0
/proc/27335/statm: 26314 25745 52 112 0 26198 0
Current children cumulated CPU time (s) 1777.68
Current children cumulated vsize (KiB) 129056



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1806.12 s]
/proc/loadavg: 2.93 3.12 2.89 3/96 28888
/proc/meminfo: memFree=15736/2055920 swapFree=4192812/4192956
[pid=27316] ppid=27314 vsize=23800 CPUtime=0.9
/proc/27316/stat : 27316 (sugar) S 27314 27316 27251 0 -1 4194304 3691 7306 0 33 18 2 62 8 16 0 1 0 30832170 24371200 3170 18446744073709551615 4194304 4206940 548682069232 18446744073709551615 244680011874 0 0 4224 16386 18446744071563648864 0 0 17 1 0 0
/proc/27316/statm: 5950 3170 373 3 0 3050 0
[pid=27326] ppid=27316 vsize=0 CPUtime=115.21
/proc/27326/stat : 27326 (java) Z 27316 27316 27251 0 -1 1036 101670 0 2 0 11358 163 0 0 16 0 1 0 30832262 0 0 18446744073709551615 0 0 0 0 0 0 4 0 23759 18446744071563351923 0 0 17 1 0 0
/proc/27326/statm: 0 0 0 0 0 0 0
[pid=27335] ppid=27316 vsize=105256 CPUtime=1683.95
/proc/27335/stat : 27335 (minisat) R 27316 27316 27251 0 -1 0 26779 0 5 0 167894 501 0 0 25 0 1 0 30843949 107782144 25745 18446744073709551615 134512640 134974460 4294956768 18446744073709551615 134528363 0 0 4096 3 0 0 0 17 1 0 0
/proc/27335/statm: 26314 25745 52 112 0 26198 0
Current children cumulated CPU time (s) 1800.06
Current children cumulated vsize (KiB) 129056

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Child ended because it received signal 9 (SIGKILL)

!!! problem with CPU time !!!
wait4(...,&childrusage) returns 27316 and gives childrusage.ru_utime.tv_sec=0 childrusage.ru_utime.tv_usec=805877 childrusage.ru_stime.tv_sec=0 childrusage.ru_stime.tv_usec=115982
CPU time returned by wait4() is 0.921859
while last known CPU time is 1800.08

Solver probably didn't/couldn't wait for its children
Using last known CPU time as value...

Real time (s): 1808.2
CPU time (s): 1800.08
CPU user time (s): 1793.32
CPU system time (s): 6.76
CPU usage (%): 99.551
Max. virtual memory (cumulated for all children) (KiB): 822060

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.805877
system time used= 0.115982
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 11018
page faults= 33
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 743
involuntary context switches= 632

runsolver used 2.09468 s user time and 5.72213 s system time

The end

Launcher Data (download as text)

Begin job on node7 on Mon Jan 29 22:12:54 UTC 2007


IDJOB= 238806
IDBENCH= 7021
IDSOLVER= 86
FILE ID= node7/238806-1170108774

PBS_JOBID= 3689968

Free space on /tmp= 50500 MiB

SOLVER NAME= sugar 0.40
BENCH NAME= HOME/pub/bench/CPAI06/blackHole/BH-4-7/BlackHole-4-7-e-5_ext.xml
COMMAND LINE= /tmp/evaluation/238806-1170108774/solver /tmp/evaluation/238806-1170108774 /tmp/evaluation/238806-1170108774/unknown.xml
RUNSOLVER COMMAND LINE= runsolver  --timestamp  -w ROOT/results/node7/watcher-238806-1170108774 -o ROOT/results/node7/solver-238806-1170108774 -C 1800 -M 900  /tmp/evaluation/238806-1170108774/solver /tmp/evaluation/238806-1170108774 /tmp/evaluation/238806-1170108774/unknown.xml

META MD5SUM SOLVER= 0a27e717dfe98d83d9802fe0eea45073
MD5SUM BENCH=  4c58eb69aa8058c4fc80a5c46de30e6e

RANDOM SEED= 395928193

TIME LIMIT= 1800 seconds

MEMORY LIMIT= 900 MiB


/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.232
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5914.62
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.232
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5586.94
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:      2055920 kB
MemFree:         66648 kB
Buffers:          9308 kB
Cached:        1729248 kB
SwapCached:          0 kB
Active:         200036 kB
Inactive:      1705892 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      2055920 kB
LowFree:         66648 kB
SwapTotal:     4192956 kB
SwapFree:      4192812 kB
Dirty:          370008 kB
Writeback:           0 kB
Mapped:         192692 kB
Slab:            67904 kB
Committed_AS:   648004 kB
PageTables:       2428 kB
VmallocTotal: 536870911 kB
VmallocUsed:    264952 kB
VmallocChunk: 536605679 kB
HugePages_Total:     0
HugePages_Free:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 47733 MiB



End job on node7 on Mon Jan 29 22:43:04 UTC 2007