Trace number 3284934

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
sathys 2011-04-01? (TO) 1300.03 1300.01

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
transport-transport-city-sequential-35nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.030-NOTKNOWN.cnf
MD5SUMfce318b20c1a474ab81b77594d3f6f99
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark583.061
Satisfiable
(Un)Satisfiability was proved
Number of variables945406
Number of clauses5079060
Sum of the clauses size12071640
Maximum clause length982
Minimum clause length1
Number of clauses of size 1720
Number of clauses of size 25035980
Number of clauses of size 3240
Number of clauses of size 4960
Number of clauses of size 52400
Number of clauses of size over 538760

Solver Data

0.00/0.00	static
0.00/0.00	HOME/instance-3284934-1303762128.cnf
0.00/0.00	c Parsing...
0.00/0.00	c num clauses = 5079060
0.00/0.00	c too many clauses .. no preprocessing
3.59/3.69	c 
3.59/3.69	c seed = 0, numatom = 945406, numclause = 42360, numliterals = 12071640
3.59/3.69	c ---------------------------------------------------------------------------------------------------------------------------------------------
4.29/4.31	c | resCDCL |    tps  |   #impl |  #learnt |  #conf |  #clean |    #propa |   #deci ||  tryLS | numflip |  maxSat/maxNoBi |  nbAlea |    Equi |
4.29/4.31	c ---------------------------------------------------------------------------------------------------------------------------------------------
326.29/326.34	c |       2 |   322.7 |  105779 |      445 |      0 |       1 |  10814687 |   93860 ||      2 |  396666 |      77/     75 |    1236 |       0 |
327.70/327.72	c |       4 |     1.4 |  106135 |      922 |    294 |       2 |  17142828 |  121543 ||      4 |  401486 |      40/     35 |    1615 |       0 |
329.60/329.67	c |       6 |     2.0 |  106136 |     1384 |    190 |       3 |  23351256 |  152329 ||      6 |  408272 |      40/     35 |    2010 |       0 |
332.00/332.01	c |       7 |     2.3 |  106138 |     1901 |    390 |       4 |  29459782 |  173342 ||      7 |  416703 |      68/     46 |    2590 |       0 |
333.90/333.96	c |      11 |     2.0 |  106139 |     2380 |    280 |       5 |  39559495 |  213063 ||     11 |  425603 |      70/     52 |    3284 |       0 |
337.00/337.03	c |      14 |     3.1 |  106170 |     2996 |    476 |       6 |  48529235 |  247848 ||     14 |  436547 |      50/     36 |    4203 |       0 |
339.91/339.96	c |      15 |     2.9 |  106172 |     3521 |    978 |       7 |  58889891 |  289376 ||     15 |  448644 |      69/     52 |    5273 |       0 |
345.30/345.37	c |      16 |     5.4 |  106175 |     4016 |     75 |       8 |  67087784 |  350916 ||     16 |  455777 |      52/     51 |    5503 |       0 |
348.70/348.73	c |      21 |     3.4 |  106177 |     4388 |     70 |       9 |  82500672 |  398502 ||     21 |  472281 |      66/     41 |    6551 |       0 |
353.60/353.65	c |      25 |     4.9 |  106179 |     5000 |    171 |      10 |  97318879 |  478823 ||     25 |  484739 |      61/     43 |    7624 |       0 |
357.60/357.60	c |      29 |     4.0 |  106239 |     5638 |    557 |      11 | 115184779 |  558445 ||     29 |  500655 |      58/     45 |    8707 |       0 |
362.00/362.09	c |      30 |     4.5 |  106242 |     6390 |    360 |      12 | 126388987 |  610595 ||     30 |  514308 |      49/     45 |    9838 |       0 |
366.10/366.11	c |      31 |     4.0 |  106458 |     7127 |   2061 |      13 | 142479233 |  653915 ||     31 |  535465 |      47/     27 |   11832 |       0 |
374.20/374.22	c |      31 |     8.1 |  106459 |     7700 |   1461 |      14 | 153365929 |  699931 ||     31 |  540647 |      47/     27 |   11950 |       0 |
380.80/380.80	c |      38 |     6.6 |  106462 |     8289 |    346 |      15 | 175458705 |  793682 ||     38 |  560196 |      48/     44 |   13495 |       0 |
386.30/386.32	c |      44 |     5.5 |  107237 |     8756 |    234 |      16 | 194951959 |  912016 ||     44 |  576883 |      45/     32 |   14655 |       0 |
390.90/390.99	c |      46 |     4.7 |  107238 |     9406 |    232 |      17 | 221430818 |  967824 ||     46 |  594799 |      45/     32 |   16279 |       0 |
398.20/398.26	c |      53 |     7.3 |  107244 |    10132 |    216 |      18 | 246130187 | 1062333 ||     53 |  614960 |      64/     46 |   17823 |       0 |
405.10/405.13	c |      59 |     6.9 |  107245 |    10794 |    104 |      19 | 269318441 | 1167200 ||     59 |  636682 |      35/     24 |   19038 |       0 |
412.40/412.43	c |      61 |     7.3 |  107247 |    11657 |    703 |      20 | 288138626 | 1219342 ||     61 |  655029 |      52/     32 |   20575 |       0 |
420.00/420.01	c |      62 |     7.6 |  107249 |    12469 |    805 |      21 | 309447616 | 1295361 ||     62 |  688786 |      40/     34 |   22647 |       0 |
433.41/433.43	c |      63 |    13.4 |  107249 |    13100 |   3411 |      22 | 325251387 | 1336475 ||     63 |  725009 |      38/     33 |   26289 |       0 |
441.01/441.03	c |      63 |     7.6 |  107260 |    13679 |   1310 |      23 | 348985312 | 1418469 ||     63 |  741978 |      38/     33 |   26665 |       0 |
450.10/450.14	c |      67 |     9.1 |  107261 |    14319 |    277 |      24 | 369528847 | 1480257 ||     67 |  751474 |      52/     41 |   27336 |       0 |
459.60/459.62	c |      76 |     9.5 |  107266 |    14910 |    260 |      25 | 399616920 | 1605101 ||     76 |  778416 |      45/     43 |   29300 |       0 |
468.51/468.59	c |      80 |     9.0 |  107272 |    15699 |     42 |      26 | 424780465 | 1715267 ||     80 |  800525 |      74/     56 |   31183 |       0 |
478.90/478.92	c |      90 |    10.3 |  107275 |    16365 |    129 |      27 | 456843112 | 1820208 ||     90 |  830987 |      56/     38 |   33290 |       0 |
489.10/489.19	c |      94 |    10.3 |  107277 |    17471 |   1020 |      28 | 483341736 | 1908916 ||     94 |  861106 |      39/     26 |   36746 |       0 |
500.30/500.34	c |      96 |    11.1 |  107279 |    18485 |     21 |      29 | 505740530 | 1982393 ||     96 |  885312 |      53/     48 |   37386 |       0 |
511.20/511.25	c |     107 |    10.9 |  107283 |    19024 |    201 |      30 | 544901210 | 2112216 ||    107 |  918723 |      61/     46 |   39769 |       0 |
524.30/524.32	c |     112 |    13.1 |  107285 |    20121 |    191 |      31 | 574160108 | 2228789 ||    112 |  949638 |      47/     29 |   41898 |       0 |
540.01/540.07	c |     123 |    15.8 |  107290 |    20906 |    571 |      32 | 609555307 | 2367249 ||    123 |  986727 |      45/     30 |   44572 |       0 |
552.40/552.43	c |     125 |    12.4 |  107290 |    21812 |     70 |      33 | 640075382 | 2456720 ||    125 | 1031398 |      31/     28 |   47672 |       0 |
564.21/564.30	c |     126 |    11.9 |  107290 |    22725 |   3871 |      34 | 668394822 | 2540830 ||    126 | 1068684 |      31/     24 |   51455 |       0 |
581.71/581.79	c |     126 |    17.5 |  107291 |    23225 |   2972 |      35 | 690287680 | 2610075 ||    126 | 1085360 |      31/     24 |   51760 |       0 |
594.91/594.91	c |     127 |    13.1 |  107299 |    24085 |   6973 |      36 | 745770077 | 2726336 ||    127 | 1157904 |      40/     32 |   59506 |       0 |
610.71/610.76	c |     127 |    15.9 |  107302 |    25473 |   1474 |      37 | 779778707 | 2836206 ||    127 | 1189602 |      40/     32 |   60117 |       0 |
620.51/620.53	c |     127 |     9.8 |  107303 |    26474 |   5677 |      38 | 806159317 | 2879986 ||    127 | 1189602 |      40/     32 |   60117 |       0 |
632.01/632.01	c |     139 |    11.5 |  107311 |    27103 |     48 |      39 | 849479447 | 3028536 ||    139 | 1222087 |      44/     40 |   62514 |       0 |
652.11/652.15	c |     148 |    20.1 |  107314 |    27576 |    133 |      40 | 898093242 | 3139273 ||    148 | 1258338 |      57/     36 |   65416 |       0 |
667.31/667.33	c |     157 |    15.2 |  107319 |    28931 |    411 |      41 | 937775986 | 3289287 ||    157 | 1301114 |      34/     22 |   68540 |       0 |
686.01/686.03	c |     162 |    18.7 |  108838 |    29852 |     88 |      42 | 969029073 | 3412905 ||    162 | 1334707 |      61/     41 |   71245 |       0 |
699.71/699.78	c |     173 |    13.8 |  108845 |    30015 |     66 |      43 | 1015228103 | 3583718 ||    173 | 1374562 |      26/     24 |   74652 |       0 |
714.51/714.55	c |     187 |    14.8 |  108856 |    31229 |    223 |      44 | 1057052487 | 3744645 ||    187 | 1411671 |      43/     29 |   77678 |       0 |
733.71/733.72	c |     189 |    19.2 |  108858 |    32737 |    917 |      45 | 1094093948 | 3851760 ||    189 | 1438235 |      30/     23 |   80531 |       0 |
759.81/759.86	c |     190 |    26.1 |  108859 |    33719 |   1116 |      46 | 1141199718 | 3966653 ||    190 | 1480850 |      40/     25 |   84585 |       0 |
781.51/781.53	c |     200 |    21.7 |  108864 |    34738 |    286 |      47 | 1186029857 | 4117975 ||    200 | 1509672 |      46/     39 |   86647 |       0 |
796.71/796.73	c |     212 |    15.2 |  108904 |    35724 |    366 |      48 | 1239362213 | 4279013 ||    212 | 1550005 |      39/     36 |   90279 |       0 |
812.01/812.09	c |     221 |    15.4 |  108907 |    37067 |   1447 |      49 | 1293709331 | 4415705 ||    221 | 1596313 |      48/     37 |   94742 |       0 |
832.11/832.13	c |     231 |    20.0 |  108909 |    38273 |    121 |      50 | 1342474959 | 4569625 ||    231 | 1637933 |      50/     38 |   97066 |       0 |
849.82/849.82	c |     243 |    17.7 |  108914 |    39358 |    494 |      51 | 1392763327 | 4722446 ||    243 | 1682433 |      35/     19 |  100727 |       0 |
869.72/869.75	c |     252 |    19.9 |  108917 |    40896 |   1874 |      52 | 1445834907 | 4872583 ||    252 | 1732867 |      39/     30 |  105187 |       0 |
887.51/887.55	c |     253 |    17.8 |  108921 |    41826 |    373 |      53 | 1498058893 | 4991376 ||    253 | 1785541 |      38/     21 |  109289 |       0 |
910.11/910.18	c |     254 |    22.6 |  108922 |    42840 |   6174 |      54 | 1536141046 | 5081675 ||    254 | 1844805 |      47/     34 |  116709 |       0 |
929.02/929.04	c |     254 |    18.9 |  108923 |    43378 |   2466 |      55 | 1582598933 | 5204070 ||    254 | 1901781 |      47/     34 |  117759 |       0 |
949.12/949.18	c |     255 |    20.2 |  108959 |    44464 |   8467 |      56 | 1625216206 | 5315939 ||    255 | 1950109 |      54/     30 |  131293 |       0 |
978.02/978.03	c |     255 |    28.8 |  108961 |    45255 |   6103 |      57 | 1686827497 | 5454443 ||    255 | 2033929 |      51/     30 |  132800 |       0 |
994.33/994.30	c |     255 |    16.3 |  108964 |    46503 |   1570 |      58 | 1751311816 | 5606003 ||    255 | 2091694 |      51/     30 |  134193 |       0 |
1010.72/1010.77	c |     255 |    16.5 |  108976 |    47922 |   7872 |      59 | 1788432424 | 5692143 ||    255 | 2091694 |      51/     30 |  134193 |       0 |
1034.12/1034.10	c |     261 |    23.3 |  108986 |    48200 |     51 |      60 | 1837517055 | 5893288 ||    261 | 2107355 |      36/     23 |  135237 |       0 |
1054.62/1054.61	c |     276 |    20.5 |  108993 |    49294 |    125 |      61 | 1896881747 | 6087856 ||    276 | 2160472 |      55/     48 |  139365 |       0 |
1075.02/1075.00	c |     286 |    20.4 |  108998 |    50150 |    104 |      62 | 1950515024 | 6255588 ||    286 | 2210302 |      31/     21 |  144291 |       0 |
1096.73/1096.71	c |     301 |    21.7 |  109005 |    51803 |    773 |      63 | 2013721703 | 6455149 ||    301 | 2260280 |      42/     31 |  148381 |       0 |
1119.12/1119.17	c |     316 |    22.5 |  109019 |    53086 |   1130 |      64 | 2078528989 | 6662416 ||    316 | 2316246 |      66/     36 |  152558 |       0 |
1143.53/1143.53	c |     318 |    24.4 |  109023 |    54194 |   4428 |      65 | 2133718287 | 6805633 ||    318 | 2386315 |      33/     26 |  158560 |       0 |
1166.12/1166.19	c |     329 |    22.7 |  109046 |    56351 |     88 |      66 | 2183230670 | 7004230 ||    329 | 2420761 |      51/     40 |  160819 |       0 |
1190.32/1190.35	c |     347 |    24.2 |  109056 |    57674 |    248 |      67 | 2255480030 | 7233535 ||    347 | 2482814 |      24/     17 |  165756 |       0 |
1213.23/1213.29	c |     356 |    22.9 |  109062 |    59429 |    224 |      68 | 2330502024 | 7415346 ||    356 | 2533201 |      57/     31 |  170162 |       0 |
1235.72/1235.74	c |     372 |    22.5 |  109069 |    60652 |    489 |      69 | 2400275057 | 7608257 ||    372 | 2597402 |      30/     22 |  174573 |       0 |
1256.82/1256.84	c |     381 |    21.1 |  109073 |    62146 |   1862 |      70 | 2473471004 | 7787284 ||    381 | 2656308 |      50/     20 |  182241 |       0 |
1287.32/1287.34	c |     382 |    30.5 |  109077 |    63237 |   4561 |      71 | 2542748004 | 7930361 ||    382 | 2725304 |      40/     25 |  189846 |       0 |
1300.03/1300.00	c |     382 |    12.7 |  109077 |    66138 |   2902 |      71 | 2566858229 | 7978381 ||    382 | 2752377 |      40/     25 |  190382 |       0 |
1300.03/1300.00	c c 
1300.03/1300.00	c -----------------------------------
1300.03/1300.00	c # temps = 1300.040000
1300.03/1300.00	c # cleaningDB = 71
1300.03/1300.00	c # clause binaire = 5036278
1300.03/1300.00	c # literal removed by litRedundant = 56930857
1300.03/1300.00	c # literal removed by litRedundantBin = 49482659
1300.03/1300.00	c # init clause removed by simplify = 7178
1300.03/1300.00	c # learnt clause removed by simplify = 215
1300.03/1300.00	c # une clause fausse était une apprise = 0
1300.03/1300.00	c # equivalence dectecte = 0
1300.03/1300.00	c # Memory used           : 734.27 MB
1300.03/1300.00	c 
1300.03/1300.00	c ########################
1300.03/1300.00	c ####   ##  ###   ## ####
1300.03/1300.00	c #### #### # ## #### ####
1300.03/1300.00	c ####   ##  ###   ##  ###
1300.03/1300.00	c ########################
1300.03/1300.00	c # restart = 382
1300.03/1300.00	c # conflit = 287046
1300.03/1300.00	c # decision = 7978381
1300.03/1300.00	c # propagation = 2566858229 	 (1974445.577828 (prop/sec))
1300.03/1300.00	c 
1300.03/1300.00	c ########################
1300.03/1300.00	c ### ### #   ##   ##   ##
1300.03/1300.00	c ### # # # #### # ### ###
1300.03/1300.00	c ###  #  ### ## # ### ###
1300.03/1300.00	c ########################
1300.03/1300.00	c # numtry = 382
1300.03/1300.00	c # numflips = 2752377
1300.03/1300.00	c # maxSat = 24
1300.03/1300.00	c # freeFlip = 190374
1300.03/1300.00	c # nbBoundary = 0
1300.03/1300.00	c 
1300.03/1300.00	=============================================================================================================================================
1300.03/1300.00	s UNKNOWN

Verifier Data

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3284934-1303762128/watcher-3284934-1303762128 -o /tmp/evaluation-result-3284934-1303762128/solver-3284934-1303762128 -C 2400 -W 1300 -M 7680 ./esg_static.sh HOME/instance-3284934-1303762128.cnf rnovelty 

running on 2 cores: 4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 7864320 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 7915520 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.77 3.95 3.90 5/188 23979
/proc/meminfo: memFree=15408052/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0

[startup+0.0831019 s]
/proc/loadavg: 3.77 3.95 3.90 5/188 23979
/proc/meminfo: memFree=15408052/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8704

[startup+0.101101 s]
/proc/loadavg: 3.77 3.95 3.90 5/188 23979
/proc/meminfo: memFree=15408052/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8704

[startup+0.301062 s]
/proc/loadavg: 3.77 3.95 3.90 5/188 23979
/proc/meminfo: memFree=15408052/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8704

[startup+0.701004 s]
/proc/loadavg: 3.77 3.95 3.90 5/188 23979
/proc/meminfo: memFree=15408052/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8704

[startup+1.50085 s]
/proc/loadavg: 3.77 3.95 3.90 5/190 23982
/proc/meminfo: memFree=15174596/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
[pid=23981] ppid=23979 vsize=223080 CPUtime=1.49 cores=4,6
/proc/23981/stat : 23981 (sathys) R 23979 23979 23613 0 -1 4202496 46408 0 0 0 142 7 0 0 25 0 1 0 487813549 228433920 46317 33554432000 4194304 4241092 140735004114224 18446744073709551615 253640520389 0 0 4096 24578 0 0 0 17 4 0 0 0
/proc/23981/statm: 55770 46317 116 12 0 54234 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 231784

[startup+3.10058 s]
/proc/loadavg: 3.77 3.95 3.90 5/196 23994
/proc/meminfo: memFree=15202736/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
[pid=23981] ppid=23979 vsize=257392 CPUtime=3.09 cores=4,6
/proc/23981/stat : 23981 (sathys) R 23979 23979 23613 0 -1 4202496 54960 0 0 0 300 9 0 0 25 0 1 0 487813549 263569408 54869 33554432000 4194304 4241092 140735004114224 18446744073709551615 253640859392 0 0 4096 24578 0 0 0 17 4 0 0 0
/proc/23981/statm: 64348 54869 117 12 0 62812 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 266096

[startup+6.30097 s]
/proc/loadavg: 4.03 4.00 3.92 5/196 23994
/proc/meminfo: memFree=14746500/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
[pid=23981] ppid=23979 vsize=553100 CPUtime=6.29 cores=4,6
/proc/23981/stat : 23981 (sathys) R 23979 23979 23613 0 -1 4202496 127241 0 0 0 611 18 0 0 25 0 1 0 487813549 566374400 127026 33554432000 4194304 4241092 140735004114224 18446744073709551615 4216815 0 0 4096 24578 0 0 0 17 4 0 0 0
/proc/23981/statm: 138275 127026 239 12 0 136739 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 561804

[startup+12.7008 s]
/proc/loadavg: 4.02 4.00 3.92 5/196 23994
/proc/meminfo: memFree=14386560/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
[pid=23981] ppid=23979 vsize=553100 CPUtime=12.69 cores=4,6
/proc/23981/stat : 23981 (sathys) R 23979 23979 23613 0 -1 4202496 127491 0 0 0 1251 18 0 0 25 0 1 0 487813549 566374400 127232 33554432000 4194304 4241092 140735004114224 18446744073709551615 4216792 0 0 4096 24578 0 0 0 17 4 0 0 0
/proc/23981/statm: 138275 127232 196 12 0 136739 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 561804

[startup+25.5005 s]
/proc/loadavg: 4.02 4.00 3.92 5/196 23995
/proc/meminfo: memFree=14372696/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
[pid=23981] ppid=23979 vsize=553100 CPUtime=25.49 cores=4,6
/proc/23981/stat : 23981 (sathys) R 23979 23979 23613 0 -1 4202496 127683 0 0 0 2531 18 0 0 25 0 1 0 487813549 566374400 127407 33554432000 4194304 4241092 140735004114224 18446744073709551615 4216815 0 0 4096 24578 0 0 0 17 4 0 0 0
/proc/23981/statm: 138275 127407 179 12 0 136739 0
Current children cumulated CPU time (s) 25.49
Current children cumulated vsize (KiB) 561804

[startup+51.1009 s]
/proc/loadavg: 4.08 4.01 3.93 5/196 23996
/proc/meminfo: memFree=14353872/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
[pid=23981] ppid=23979 vsize=553100 CPUtime=51.09 cores=4,6
/proc/23981/stat : 23981 (sathys) R 23979 23979 23613 0 -1 4202496 128147 0 0 0 5091 18 0 0 25 0 1 0 487813549 566374400 127836 33554432000 4194304 4241092 140735004114224 18446744073709551615 4216792 0 0 4096 24578 0 0 0 17 4 0 0 0
/proc/23981/statm: 138275 127836 144 12 0 136739 0
Current children cumulated CPU time (s) 51.09
Current children cumulated vsize (KiB) 561804

[startup+102.308 s]
/proc/loadavg: 4.04 4.01 3.93 5/196 24034
/proc/meminfo: memFree=14445324/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
[pid=23981] ppid=23979 vsize=553100 CPUtime=102.3 cores=4,6
/proc/23981/stat : 23981 (sathys) R 23979 23979 23613 0 -1 4202496 128541 0 0 0 10212 18 0 0 25 0 1 0 487813549 566374400 128209 33554432000 4194304 4241092 140735004114224 18446744073709551615 4216804 0 0 4096 24578 0 0 0 17 4 0 0 0
/proc/23981/statm: 138275 128209 165 12 0 136739 0
Current children cumulated CPU time (s) 102.3
Current children cumulated vsize (KiB) 561804

[startup+162.301 s]
/proc/loadavg: 4.06 4.02 3.93 5/196 24035
/proc/meminfo: memFree=14389580/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
[pid=23981] ppid=23979 vsize=553100 CPUtime=162.29 cores=4,6
/proc/23981/stat : 23981 (sathys) R 23979 23979 23613 0 -1 4202496 128822 0 0 0 16211 18 0 0 25 0 1 0 487813549 566374400 128480 33554432000 4194304 4241092 140735004114224 18446744073709551615 4216815 0 0 4096 24578 0 0 0 17 4 0 0 0
/proc/23981/statm: 138275 128480 189 12 0 136739 0
Current children cumulated CPU time (s) 162.29
Current children cumulated vsize (KiB) 561804

[startup+222.301 s]
/proc/loadavg: 4.02 4.01 3.93 5/196 24037
/proc/meminfo: memFree=14346488/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
[pid=23981] ppid=23979 vsize=553100 CPUtime=222.29 cores=4,6
/proc/23981/stat : 23981 (sathys) R 23979 23979 23613 0 -1 4202496 129198 0 0 0 22211 18 0 0 25 0 1 0 487813549 566374400 128837 33554432000 4194304 4241092 140735004114224 18446744073709551615 4218232 0 0 4096 24578 0 0 0 17 4 0 0 0
/proc/23981/statm: 138275 128837 231 12 0 136739 0
Current children cumulated CPU time (s) 222.29
Current children cumulated vsize (KiB) 561804

[startup+282.301 s]
/proc/loadavg: 4.15 4.04 3.94 5/196 24069
/proc/meminfo: memFree=14343948/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
[pid=23981] ppid=23979 vsize=553232 CPUtime=282.29 cores=4,6
/proc/23981/stat : 23981 (sathys) R 23979 23979 23613 0 -1 4202496 129393 0 0 0 28211 18 0 0 25 0 1 0 487813549 566509568 128976 33554432000 4194304 4241092 140735004114224 18446744073709551615 4216815 0 0 4096 24578 0 0 0 17 4 0 0 0
/proc/23981/statm: 138308 128976 214 12 0 136772 0
Current children cumulated CPU time (s) 282.29
Current children cumulated vsize (KiB) 561936

[startup+342.301 s]
/proc/loadavg: 4.05 4.03 3.94 5/196 24070
/proc/meminfo: memFree=14329560/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
[pid=23981] ppid=23979 vsize=565016 CPUtime=342.3 cores=4,6
/proc/23981/stat : 23981 (sathys) R 23979 23979 23613 0 -1 4202496 132692 0 0 0 34211 19 0 0 25 0 1 0 487813549 578576384 132143 33554432000 4194304 4241092 140735004114224 18446744073709551615 4221877 0 0 4096 24578 0 0 0 17 4 0 0 0
/proc/23981/statm: 141254 132143 150 12 0 139718 0
Current children cumulated CPU time (s) 342.3
Current children cumulated vsize (KiB) 573720

[startup+402.301 s]
/proc/loadavg: 4.02 4.02 3.94 5/196 24072
/proc/meminfo: memFree=14294456/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6

################
# More data... #
################

/proc/23981/statm: 164277 155212 147 12 0 162741 0
Current children cumulated CPU time (s) 582.31
Current children cumulated vsize (KiB) 665812

[startup+642.3 s]
/proc/loadavg: 4.06 4.03 3.95 5/196 24078
/proc/meminfo: memFree=14154460/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
[pid=23981] ppid=23979 vsize=657140 CPUtime=642.31 cores=4,6
/proc/23981/stat : 23981 (sathys) R 23979 23979 23613 0 -1 4202496 156475 0 0 0 64208 23 0 0 25 0 1 0 487813549 672911360 155220 33554432000 4194304 4241092 140735004114224 18446744073709551615 4221357 0 0 4096 24578 0 0 0 17 4 0 0 0
/proc/23981/statm: 164285 155220 147 12 0 162749 0
Current children cumulated CPU time (s) 642.31
Current children cumulated vsize (KiB) 665844

[startup+702.3 s]
/proc/loadavg: 4.02 4.02 3.95 5/196 24080
/proc/meminfo: memFree=14099372/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
[pid=23981] ppid=23979 vsize=660288 CPUtime=702.31 cores=4,6
/proc/23981/stat : 23981 (sathys) R 23979 23979 23613 0 -1 4202496 157268 0 0 0 70208 23 0 0 25 0 1 0 487813549 676134912 156013 33554432000 4194304 4241092 140735004114224 18446744073709551615 4223705 0 0 4096 24578 0 0 0 17 4 0 0 0
/proc/23981/statm: 165072 156013 147 12 0 163536 0
Current children cumulated CPU time (s) 702.31
Current children cumulated vsize (KiB) 668992

[startup+762.307 s]
/proc/loadavg: 4.01 4.02 3.95 5/196 24081
/proc/meminfo: memFree=14087700/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
[pid=23981] ppid=23979 vsize=672060 CPUtime=762.31 cores=4,6
/proc/23981/stat : 23981 (sathys) R 23979 23979 23613 0 -1 4202496 160194 0 0 0 76208 23 0 0 25 0 1 0 487813549 688189440 158939 33554432000 4194304 4241092 140735004114224 18446744073709551615 4222067 0 0 4096 24578 0 0 0 17 4 0 0 0
/proc/23981/statm: 168015 158939 147 12 0 166479 0
Current children cumulated CPU time (s) 762.31
Current children cumulated vsize (KiB) 680764

[startup+822.301 s]
/proc/loadavg: 4.00 4.01 3.95 5/196 24083
/proc/meminfo: memFree=14054224/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
[pid=23981] ppid=23979 vsize=672092 CPUtime=822.31 cores=4,6
/proc/23981/stat : 23981 (sathys) R 23979 23979 23613 0 -1 4202496 160201 0 0 0 82208 23 0 0 25 0 1 0 487813549 688222208 158946 33554432000 4194304 4241092 140735004114224 18446744073709551615 4222310 0 0 4096 24578 0 0 0 17 4 0 0 0
/proc/23981/statm: 168023 158946 147 12 0 166487 0
Current children cumulated CPU time (s) 822.31
Current children cumulated vsize (KiB) 680796

[startup+882.301 s]
/proc/loadavg: 4.00 4.01 3.95 5/196 24085
/proc/meminfo: memFree=14026244/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
[pid=23981] ppid=23979 vsize=688832 CPUtime=882.31 cores=4,6
/proc/23981/stat : 23981 (sathys) R 23979 23979 23613 0 -1 4202496 164386 0 0 0 88207 24 0 0 25 0 1 0 487813549 705363968 163131 33554432000 4194304 4241092 140735004114224 18446744073709551615 4222092 0 0 4096 24578 0 0 0 17 4 0 0 0
/proc/23981/statm: 172208 163131 147 12 0 170672 0
Current children cumulated CPU time (s) 882.31
Current children cumulated vsize (KiB) 697536

[startup+942.301 s]
/proc/loadavg: 4.00 4.00 3.95 5/196 24086
/proc/meminfo: memFree=13972844/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
[pid=23981] ppid=23979 vsize=698096 CPUtime=942.32 cores=4,6
/proc/23981/stat : 23981 (sathys) R 23979 23979 23613 0 -1 4202496 166738 0 0 0 94207 25 0 0 25 0 1 0 487813549 714850304 165483 33554432000 4194304 4241092 140735004114224 18446744073709551615 4228988 0 0 4096 24578 0 0 0 17 4 0 0 0
/proc/23981/statm: 174524 165483 147 12 0 172988 0
Current children cumulated CPU time (s) 942.32
Current children cumulated vsize (KiB) 706800

[startup+1002.3 s]
/proc/loadavg: 4.00 4.00 3.95 5/196 24088
/proc/meminfo: memFree=13927180/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
[pid=23981] ppid=23979 vsize=713352 CPUtime=1002.32 cores=4,6
/proc/23981/stat : 23981 (sathys) R 23979 23979 23613 0 -1 4202496 170539 0 0 0 100206 26 0 0 25 0 1 0 487813549 730472448 169284 33554432000 4194304 4241092 140735004114224 18446744073709551615 4222040 0 0 4096 24578 0 0 0 17 4 0 0 0
/proc/23981/statm: 178338 169284 147 12 0 176802 0
Current children cumulated CPU time (s) 1002.32
Current children cumulated vsize (KiB) 722056

[startup+1062.3 s]
/proc/loadavg: 4.00 4.00 3.95 5/196 24089
/proc/meminfo: memFree=13926308/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
[pid=23981] ppid=23979 vsize=713384 CPUtime=1062.32 cores=4,6
/proc/23981/stat : 23981 (sathys) R 23979 23979 23613 0 -1 4202496 170547 0 0 0 106206 26 0 0 25 0 1 0 487813549 730505216 169290 33554432000 4194304 4241092 140735004114224 18446744073709551615 4222347 0 0 4096 24578 0 0 0 17 4 0 0 0
/proc/23981/statm: 178346 169290 147 12 0 176810 0
Current children cumulated CPU time (s) 1062.32
Current children cumulated vsize (KiB) 722088

[startup+1122.3 s]
/proc/loadavg: 4.10 4.03 3.96 5/196 24091
/proc/meminfo: memFree=13871312/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
[pid=23981] ppid=23979 vsize=713384 CPUtime=1122.33 cores=4,6
/proc/23981/stat : 23981 (sathys) R 23979 23979 23613 0 -1 4202496 170555 0 0 0 112207 26 0 0 25 0 1 0 487813549 730505216 169296 33554432000 4194304 4241092 140735004114224 18446744073709551615 4222040 0 0 4096 24578 0 0 0 17 4 0 0 0
/proc/23981/statm: 178346 169296 147 12 0 176810 0
Current children cumulated CPU time (s) 1122.33
Current children cumulated vsize (KiB) 722088

[startup+1182.3 s]
/proc/loadavg: 4.03 4.02 3.96 5/196 24093
/proc/meminfo: memFree=13801244/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
[pid=23981] ppid=23979 vsize=741188 CPUtime=1182.32 cores=4,6
/proc/23981/stat : 23981 (sathys) R 23979 23979 23613 0 -1 4202496 177487 0 0 0 118205 27 0 0 25 0 1 0 487813549 758976512 176228 33554432000 4194304 4241092 140735004114224 18446744073709551615 4211605 0 0 4096 24578 0 0 0 17 4 0 0 0
/proc/23981/statm: 185297 176228 147 12 0 183761 0
Current children cumulated CPU time (s) 1182.32
Current children cumulated vsize (KiB) 749892

[startup+1242.3 s]
/proc/loadavg: 4.01 4.02 3.96 5/196 24094
/proc/meminfo: memFree=13793820/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
[pid=23981] ppid=23979 vsize=741220 CPUtime=1242.32 cores=4,6
/proc/23981/stat : 23981 (sathys) R 23979 23979 23613 0 -1 4202496 177498 0 0 0 124205 27 0 0 25 0 1 0 487813549 759009280 176237 33554432000 4194304 4241092 140735004114224 18446744073709551615 4222104 0 0 4096 24578 0 0 0 17 4 0 0 0
/proc/23981/statm: 185305 176237 147 12 0 183769 0
Current children cumulated CPU time (s) 1242.32
Current children cumulated vsize (KiB) 749924



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1300 s]
/proc/loadavg: 3.92 3.99 3.96 4/189 24113
/proc/meminfo: memFree=14116224/32951124 swapFree=67111528/67111528
[pid=23979] ppid=23977 vsize=8704 CPUtime=0 cores=4,6
/proc/23979/stat : 23979 (esg_static.sh) S 23977 23979 23613 0 -1 4202496 358 112 0 0 0 0 0 0 22 0 1 0 487813548 8912896 256 33554432000 4194304 4922060 140736669587776 18446744073709551615 253640678533 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/23979/statm: 2176 256 211 178 0 69 0
[pid=23981] ppid=23979 vsize=751884 CPUtime=1300.03 cores=4,6
/proc/23981/stat : 23981 (sathys) R 23979 23979 23613 0 -1 4202496 180173 0 0 0 129975 28 0 0 25 0 1 0 487813549 769929216 178912 33554432000 4194304 4241092 140735004114224 18446744073709551615 4223077 0 0 4096 24578 0 0 0 17 4 0 0 0
/proc/23981/statm: 187971 178912 147 12 0 186435 0
Current children cumulated CPU time (s) 1300.03
Current children cumulated vsize (KiB) 760588

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 23979 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=0
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=1999
# CPU time returned by wait4() is 0.001999
# while last known CPU time is 1300.03
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1300.01
CPU time (s): 1300.03
CPU user time (s): 1299.75
CPU system time (s): 0.28
CPU usage (%): 100.001
Max. virtual memory (cumulated for all children) (KiB): 760588

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0
system time used= 0.001999
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 470
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 6
involuntary context switches= 4

runsolver used 2.35364 second user time and 4.75628 second system time

The end

Launcher Data

Begin job on node106 at 2011-04-25 22:08:52
IDJOB=3284934
IDBENCH=82367
IDSOLVER=1610
FILE ID=node106/3284934-1303762128
RUNJOBID= node106-1303762128-23942
PBS_JOBID= 13157143
Free space on /tmp= 71480 MiB

SOLVER NAME= sathys 2011-04-01
BENCH NAME= SAT11/application/rintanen/SATPlanning/transport-transport-city-sequential-35nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.030-NOTKNOWN.cnf
COMMAND LINE= ./esg_static.sh BENCHNAME rnovelty
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3284934-1303762128/watcher-3284934-1303762128 -o /tmp/evaluation-result-3284934-1303762128/solver-3284934-1303762128 -C 2400 -W 1300 -M 7680  ./esg_static.sh HOME/instance-3284934-1303762128.cnf rnovelty

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 7680 MiB
NBCORE= 2

MD5SUM BENCH= fce318b20c1a474ab81b77594d3f6f99
RANDOM SEED=762643249

node106.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.20
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      15408448 kB
Buffers:       1988004 kB
Cached:       14594368 kB
SwapCached:          0 kB
Active:       10140064 kB
Inactive:      6830004 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      15408448 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:          295264 kB
Writeback:           0 kB
AnonPages:      387232 kB
Mapped:          15740 kB
Slab:           509024 kB
PageTables:       6168 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   616604 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71472 MiB
End job on node106 at 2011-04-25 22:30:34