Trace number 4396870

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
cosoco 2.0? (TO) 2519.9 2520.01

General information on the benchmark

NameQuasiGroups/QuasiGroups-elt-qg3/
QuasiGroup-3-12.xml
MD5SUMe29963f3eb1abf29691517033ab1fcf0
Bench CategoryCSP (decision problem)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark10.0419
Satisfiable
(Un)Satisfiability was proved
Number of variables288
Number of constraints266
Number of domains2
Minimum domain size12
Maximum domain size144
Distribution of domain sizes[{"size":12,"count":144},{"size":144,"count":132}]
Minimum variable degree0
Maximum variable degree135
Distribution of variable degrees[{"degree":0,"count":12},{"degree":2,"count":132},{"degree":134,"count":12},{"degree":135,"count":132}]
Minimum constraint arity3
Maximum constraint arity145
Distribution of constraint arities[{"arity":3,"count":132},{"arity":12,"count":1},{"arity":144,"count":1},{"arity":145,"count":132}]
Number of extensional constraints0
Number of intensional constraints132
Distribution of constraint types[{"type":"intension","count":132},{"type":"allDifferent","count":1},{"type":"instantiation","count":1},{"type":"element","count":132}]
Optimization problemNO
Type of objective

Solver Data 

0.00/0.03	c
0.00/0.03	c This is cosoco 2.00 --  
0.00/0.03	c
0.00/0.03	c command line: cosoco -lc -model -restarts -stick HOME/instance-4396870-1568585763.xml 
0.00/0.03	c intension  eq(add(mul(x[0][1],12),x[1][0]),y[0][1]) -> extension (support ? 1) - 144tuples
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c inc root propagations     : 2620        
0.00/0.08	c restarts enabled, print stats at every restarts
0.00/0.08	c
0.00/0.08	c
0.00/0.08	tension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c ========================================[ Problem Statistics ]===========================================
0.00/0.08	c |                                                                                                       
0.00/0.08	c |  Parse time        :         0.08 s 
0.00/0.08	c |
0.00/0.08	c |               Variables: 288
0.00/0.08	c |            Domain Sizes: 12..144
0.00/0.08	c |
0.00/0.08	c |             Constraints: 300
0.00/0.08	c |                   Arity: 1..145  -- Unary: 12  -- Ternary: 132
0.00/0.08	c | 
0.00/0.08	c |                   Types: 
0.00/0.08	c |                          All Different: 24
0.00/0.08	c |                          Element Constant: 132
0.00/0.08	c |                          Extension: 132  (nb tuples: 144..144) -- (shared: 131)
0.00/0.08	c |                          Unary: 12
0.00/0.08	c |                          
0.00/0.08	c =========================================================================================================
0.00/0.08	c conflicts      decisions      filterCalls    propagations   Root Prop.     Nogoods        min/max/avg depth
0.00/0.08	c remove useless variable y[0][0]
0.00/0.08	c remove useless variable y[1][1]
0.00/0.08	c remove useless variable y[2][2]
0.00/0.08	c remove useless variable y[3][3]
0.00/0.08	c remove useless variable y[4][4]
0.00/0.08	c remove useless variable y[5][5]
0.00/0.08	c remove useless variable y[6][6]
0.00/0.08	c remove useless variable y[7][7]
0.00/0.08	c remove useless variable y[8][8]
0.00/0.08	c remove useless variable y[9][9]
0.00/0.08	c remove useless variable y[10][10]
0.00/0.08	c remove useless variable y[11][11]
0.10/0.21	c 101            541            425075         181085         186            0              122   139   126
0.31/0.32	c 211            1019           784489         346644         186            0              20   89   76   
0.41/0.44	c 332            1467           1080663        465831         186            0              30   78   67   
0.60/0.61	c 465            2176           1706185        693253         186            0              18   88   74   
0.69/0.78	c 611            2895           2281978        926295         186            0              22   94   75   
0.89/0.97	c 771            3667           2837023        1180888        186            0              40   84   70   
1.19/1.20	c 947            4570           3632605        1521982        186            0              68   89   77   
1.39/1.43	c 1140           5501           4405314        1843311        186            0              60   81   71   
1.59/1.65	c 1352           6615           4997104        2053649        186            0              34   80   62   
1.89/1.90	c 1585           7465           5650698        2345955        186            0              18   78   64   
2.10/2.16	c 1841           8493           6323747        2623225        186            0              25   83   61   
2.49/2.54	c 2122           10225          7517835        3229885        186            0              24   94   71   
2.89/2.90	c 2431           11749          8761737        3783164        186            0              30   86   75   
3.29/3.33	c 2770           13152          10054256       4423133        186            0              32   82   67   
3.79/3.83	c 3142           15030          11678471       5218570        186            0              25   92   74   
4.19/4.29	c 3551           17023          13321728       5936190        186            0              68   94   78   
4.79/4.88	c 4000           19181          15229345       6735871        186            0              41   82   66   
5.49/5.58	c 4493           21684          17440253       7755307        186            0              32   89   67   
6.39/6.41	c 5035           24970          20085710       8949740        186            0              56   89   70   
7.00/7.05	c 5631           27671          22551931       9964841        186            0              30   95   80   
8.00/8.03	c 6286           31517          25605109       11468062       186            0              32   88   68   
8.79/8.81	c 7006           34621          28172348       12788291       186            0              45   86   75   
9.59/9.66	c 7798           38351          31131950       14135099       186            0              64   88   75   
10.79/10.80	c 8669           42869          34763103       15766110       186            0              57   90   69   
11.89/11.96	c 9627           47386          39010599       17716584       186            0              65   92   76   
13.19/13.20	c 10680          52380          43676689       19630650       186            0              55   91   77   
14.59/14.66	c 11838          59080          48738000       21909837       186            0              30   97   78   
15.89/15.96	c 13111          64599          53501204       23946439       186            0              67   91   79   
17.19/17.29	c 14511          70293          57935868       25660570       186            0              34   86   68   
18.89/18.96	c 16051          77026          62975134       27740751       186            0              20   85   66   
20.79/20.89	c 17745          85271          70179248       30540107       186            0              18   96   75   
23.39/23.44	c 19608          94061          78040943       34473565       186            0              18   91   70   
25.89/25.91	c 21657          103499         87044237       38543107       186            0              17   93   77   
28.89/28.92	c 23910          115599         97982261       43138818       186            0              38   93   74   
31.59/31.68	c 26388          127738         107407481      47028290       186            0              28   90   71   
34.39/34.47	c 29113          139447         118644766      51291005       186            0              68   97   77   
37.69/37.75	c 32110          152917         131279695      56157423       186            0              53   95   77   
41.89/41.96	c 35406          167801         143557661      62363338       186            0              27   91   69   
45.79/45.86	c 39031          185109         158348870      68427681       186            0              37   96   77   
50.19/50.26	c 43018          204284         175978072      75582918       186            0              29   94   77   
55.89/55.98	c 47403          225210         193415060      83695282       186            0              37   88   67   
61.49/61.57	c 52226          246234         213664882      92870670       186            0              37   90   75   
67.29/67.30	c 57531          269749         233781993      101184472      186            0              53   90   68   
73.59/73.68	c 63366          297614         259429303      111138835      186            0              23   99   78   
81.28/81.34	c 69784          330835         286434292      121868955      186            0              31   95   72   
91.28/91.31	c 76843          363646         317445504      137439928      186            0              51   93   70   
100.28/100.35	c 84607          401690         350997931      152083438      186            0              22   95   78   
109.78/109.81	c 93147          442292         384456184      168290883      186            0              49   96   76   
121.38/121.49	c 102541         492558         424829155      186299241      186            0              38   94   72   
133.88/133.92	c 112874         541301         465098870      204214375      186            0              39   87   69   
145.68/145.79	c 124240         594167         513690367      222342567      186            0              23   94   77   
158.38/158.41	c 136742         649823         551050120      237696014      186            0              18   90   68   
175.78/175.83	c 150494         725737         613581419      265117662      186            0              15   97   75   
192.88/192.97	c 165621         786348         664369331      288245576      186            0              42   87   67   
210.99/211.04	c 182260         869861         734800394      317701064      186            0              55   97   76   
231.78/231.85	c 200562         948646         800600395      344257343      186            0              40   91   66   
254.68/254.75	c 220694         1044575        883063301      377920386      186            0              53   97   72   
279.88/279.94	c 242839         1143004        968342409      417392889      186            0              51   94   74   
305.67/305.75	c 267198         1251178        1062917045     462315049      186            0              24   96   78   
342.17/342.27	c 293992         1384784        1178149675     519949026      186            0              43   95   70   
373.47/373.56	c 323465         1529830        1308954909     570805047      186            0              20   101   79  
412.17/412.24	c 355885         1688259        1441970910     630519129      186            0              29   93   72   
461.47/461.59	c 391547         1869519        1602251478     709948241      186            0              52   96   70   
505.77/505.88	c 430775         2054139        1772683200     779258384      186            0              37   96   77   
552.26/552.33	c 473925         2246992        1934477785     845490405      186            0              30   93   72   
617.76/617.89	c 521390         2452783        2134829461     949248677      186            0              31   94   68   
676.46/676.51	c 573601         2694183        2355976333     1041542708     186            0              37   99   77   
735.86/736.00	c 631033         2952449        2587646030     1138891824     186            0              23   100   77  
806.15/806.29	c 694208         3243985        2859541796     1252532519     186            0              27   99   78   
879.15/879.22	c 763700         3570851        3132228224     1363969470     186            0              26   96   74   
966.85/966.95	c 840141         3918020        3457682805     1501931620     186            0              39   95   76   
1066.34/1066.45	c 924226         4318230        3815647108     1642809285     186            0              52   94   70   
1188.34/1188.45	c 1016719        4733685        4201780794     1819874939     186            0              23   95   69   
1314.94/1315.08	c 1118461        5194338        4620835989     2014290032     186            0              27   95   69   
1437.63/1437.77	c 1230377        5705574        5045431972     2194694785     186            0              31   98   71   
1582.82/1582.90	c 1353484        6283854        5574225090     2424566368     186            0              36   99   76   
1752.82/1752.91	c 1488901        6918029        6146226635     2679602197     186            0              50   98   72   
1916.31/1916.48	c 1637859        7629500        6782082645     2925988608     186            0              26   98   75   
2129.01/2129.19	c 1801712        8455632        7493301038     3247491040     186            0              47   93   68   
2352.29/2352.44	c 1981950        9322755        8205694559     3567700982     186            0              38   97   68   
2519.89/2520.00	
2519.89/2520.00	c *** INTERRUPTED ***
2519.89/2520.00	c restarts              : 80          
2519.89/2520.00	c conflicts             : 2133250      (847 /sec)
2519.89/2520.00	c decisions             : 10018316     (3976 /sec)
2519.89/2520.00	c propagations          : 3830981103  (1520295 /sec)
2519.89/2520.00	c root propagations     : 2620        
2519.89/2520.00	c filter calls          : 8846004308     (3510468 /sec)
2519.89/2520.00	c useless filter calls  : 7587595619     (85 %)
2519.89/2520.00	c Memory used           : 29.93 MB
2519.89/2520.00	c CPU time              : 2519.89 s
2519.89/2520.00	c real time             : 2520 s
2519.89/2520.00	
2519.89/2520.00	s UNKNOWN

Verifier Data


Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (14045 MiB free)
  memory of node 1: 16384 MiB (15522 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4396870-1568585763/watcher-4396870-1568585763 -o /tmp/evaluation-result-4396870-1568585763/solver-4396870-1568585763 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node150-1568572744-19032 --watchdog 2580 cosoco -lc -model -restarts -stick HOME/instance-4396870-1568585763.xml 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=21599, runsolver pid=21596

[startup+0.100142 s]*
/proc/loadavg: 2.00 2.01 2.05 3/201 21600
/proc/meminfo: memFree=30273616/32770624 swapFree=67025828/67108860
[pid=21599] ppid=21596 vsize=30516 memory=6976 CPUtime=0.1 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 2046 0 0 0 10 0 0 0 20 0 1 0 434848911 31248384 1744 33554432000 4194304 6211092 140726506610528 140726506607064 5099210 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7629 1744 861 493 0 934 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 30516 KiB
Current children cumulated memory: 6976 KiB

[startup+0.209685 s]*
/proc/loadavg: 2.00 2.01 2.05 3/201 21600
/proc/meminfo: memFree=30273616/32770624 swapFree=67025828/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7240 CPUtime=0.21 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 2070 0 0 0 21 0 0 0 20 0 1 0 434848911 31383552 1810 33554432000 4194304 6211092 140726506610528 140726506607064 5397226 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1810 881 493 0 967 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7240 KiB

[startup+0.309063 s]*
/proc/loadavg: 2.00 2.01 2.05 3/201 21600
/proc/meminfo: memFree=30273616/32770624 swapFree=67025828/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7240 CPUtime=0.31 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 2071 0 0 0 31 0 0 0 20 0 1 0 434848911 31383552 1810 33554432000 4194304 6211092 140726506610528 140726506607064 5099201 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1810 881 493 0 967 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7240 KiB

[startup+0.700274 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 21600
/proc/meminfo: memFree=30273616/32770624 swapFree=67025828/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7240 CPUtime=0.69 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 2071 0 0 0 69 0 0 0 20 0 1 0 434848911 31383552 1810 33554432000 4194304 6211092 140726506610528 140726506608488 5009142 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1810 881 493 0 967 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7240 KiB

[startup+1.50027 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 21600
/proc/meminfo: memFree=30273244/32770624 swapFree=67025828/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7240 CPUtime=1.49 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 2072 0 0 0 149 0 0 0 20 0 1 0 434848911 31383552 1810 33554432000 4194304 6211092 140726506610528 140726506608488 5193696 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1810 881 493 0 967 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7240 KiB

[startup+3.109 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 21600
/proc/meminfo: memFree=30273244/32770624 swapFree=67025828/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=3.1 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 2949 0 0 0 310 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5099162 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+6.30026 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 21600
/proc/meminfo: memFree=30272888/32770624 swapFree=67025828/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=6.29 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 3828 0 0 0 629 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5099313 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+12.7003 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 21600
/proc/meminfo: memFree=30272548/32770624 swapFree=67025828/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=12.69 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 4706 0 0 0 1269 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5099111 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+25.5003 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 21600
/proc/meminfo: memFree=30272488/32770624 swapFree=67025828/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=25.49 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 5956 0 0 0 2549 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5193728 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 25.49 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+51.109 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 21603
/proc/meminfo: memFree=30272548/32770624 swapFree=67025828/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=51.09 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 6329 0 0 0 5109 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5193696 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 51.09 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+102.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 21607
/proc/meminfo: memFree=30272192/32770624 swapFree=67025828/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=102.28 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 6700 0 0 0 10228 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5039189 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 102.28 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+162.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 21672
/proc/meminfo: memFree=30264896/32770624 swapFree=67025828/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=162.28 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 7071 0 0 0 16228 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5081375 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 162.28 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+222.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 21678
/proc/meminfo: memFree=30263020/32770624 swapFree=67025828/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=222.28 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 7442 0 0 0 22228 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5035312 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 222.28 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 21690
/proc/meminfo: memFree=30262732/32770624 swapFree=67025836/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=282.28 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 7813 0 0 0 28228 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5099162 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 282.28 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 21694
/proc/meminfo: memFree=30263512/32770624 swapFree=67025836/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=342.27 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 8183 0 0 0 34227 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5397454 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 342.27 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 21700
/proc/meminfo: memFree=30263372/32770624 swapFree=67025836/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=402.27 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 8554 0 0 0 40227 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5389955 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 402.27 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 21704
/proc/meminfo: memFree=30262884/32770624 swapFree=67025836/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=462.27 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 8924 0 0 0 46227 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5039145 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 462.27 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB


################
# More data... #
################

/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 1722.22 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+1782.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 21822
/proc/meminfo: memFree=30261564/32770624 swapFree=67025856/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=1782.22 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 16886 0 0 0 178222 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5099162 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 1782.22 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+1842.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 21828
/proc/meminfo: memFree=30261272/32770624 swapFree=67025856/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=1842.21 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 17231 0 0 0 184221 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5193696 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 1842.21 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+1902.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 21832
/proc/meminfo: memFree=30261260/32770624 swapFree=67025856/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=1902.21 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 17576 0 0 0 190221 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5081333 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 1902.21 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+1962.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 21896
/proc/meminfo: memFree=30268388/32770624 swapFree=67025856/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=1962.21 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 17949 0 0 0 196221 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5193696 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 1962.21 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+2022.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 21902
/proc/meminfo: memFree=30267732/32770624 swapFree=67025856/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=2022.2 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 18297 0 0 0 202220 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5186228 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 2022.2 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+2082.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 21913
/proc/meminfo: memFree=30266828/32770624 swapFree=67025872/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=2082.2 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 18645 0 0 0 208220 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5080275 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 2082.2 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+2142.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 21917
/proc/meminfo: memFree=30267376/32770624 swapFree=67025872/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=2142.2 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 19015 0 0 0 214220 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5193696 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 2142.2 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+2202.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 21923
/proc/meminfo: memFree=30267404/32770624 swapFree=67025872/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=2202.2 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 19364 0 0 0 220220 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5099197 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 2202.2 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+2262.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 21927
/proc/meminfo: memFree=30266896/32770624 swapFree=67025872/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=2262.19 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 19713 0 0 0 226219 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5099274 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 2262.19 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+2322.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 21931
/proc/meminfo: memFree=30267456/32770624 swapFree=67025872/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=2322.19 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 20062 0 0 0 232219 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5078752 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 2322.19 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+2382.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 21938
/proc/meminfo: memFree=30267348/32770624 swapFree=67025872/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=2382.19 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 20432 0 0 0 238219 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5390560 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 2382.19 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+2442.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 21942
/proc/meminfo: memFree=30267320/32770624 swapFree=67025872/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=2442.18 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 20782 0 0 0 244218 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5193728 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 2442.18 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

[startup+2502.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 21946
/proc/meminfo: memFree=30267668/32770624 swapFree=67025872/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=2502.18 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 21132 0 0 0 250218 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5390560 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 2502.18 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+2520 s]*
/proc/loadavg: 2.00 2.01 2.05 3/200 21950
/proc/meminfo: memFree=30267244/32770624 swapFree=67025872/67108860
[pid=21599] ppid=21596 vsize=30648 memory=7316 CPUtime=2519.89 cores=0,2,4,6
/proc/21599/stat : 21599 (cosoco) R 21596 21599 19001 0 -1 4202496 21481 0 0 0 251989 0 0 0 20 0 1 0 434848911 31383552 1829 33554432000 4194304 6211092 140726506610528 140726506608488 5099120 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 34504704 140726506615819 140726506615921 140726506615921 140726506618830 0
/proc/21599/statm: 7662 1829 881 493 0 967 0
Current children cumulated CPU time: 2519.89 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7316 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

[startup+2520.01 s]
# the end of solver process 21599 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2519.89 s, system=0.009999 s

Solver just ended.
??? end of watcher thread
Dumping a history of the last processes samples

Child status: 1
Real time (s): 2520.01
CPU time (s): 2519.9
CPU user time (s): 2519.89
CPU system time (s): 0.009999
CPU usage (%): 99.9958
Max. virtual memory (cumulated for all children) (KiB): 30648
Max. memory (cumulated for all children) (KiB): 7316

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2519.89
system time used= 0.009999
maximum resident set size= 7380
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 21532
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 15
involuntary context switches= 660


# summary of solver processes directly reported to runsolver:
#   pid: 21599
#   total CPU time (s): 2519.9
#   total CPU user time (s): 2519.89
#   total CPU system time (s): 0.009999

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 7.87515 second user time and 17.1108 second system time

The end

Launcher Data 

Begin job on node150 at 2019-09-16 00:15:23
IDJOB=4396870
IDBENCH=133813
IDSOLVER=2848
FILE ID=node150/4396870-1568585763
RUNJOBID= node150-1568572744-19032
PBS_JOBID= 21703307
Free space on /tmp= 59368 MiB

SOLVER NAME= cosoco 2.0
BENCH NAME= XCSP17/QuasiGroups/QuasiGroups-elt-qg3/QuasiGroup-3-12.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4396870-1568585763/watcher-4396870-1568585763 -o /tmp/evaluation-result-4396870-1568585763/solver-4396870-1568585763 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node150-1568572744-19032 --watchdog 2580  cosoco -lc -model -restarts -stick HOME/instance-4396870-1568585763.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= e29963f3eb1abf29691517033ab1fcf0
RANDOM SEED=329462617

node150.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.79
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.79
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.79
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.79
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        30277840 kB
MemAvailable:   31842684 kB
Buffers:          173372 kB
Cached:          1652148 kB
SwapCached:        12056 kB
Active:           826680 kB
Inactive:        1043564 kB
Active(anon):       5832 kB
Inactive(anon):    76960 kB
Active(file):     820848 kB
Inactive(file):   966604 kB
Unevictable:      110960 kB
Mlocked:          110960 kB
SwapTotal:      67108860 kB
SwapFree:       67025828 kB
Dirty:              2792 kB
Writeback:             0 kB
AnonPages:        181336 kB
Mapped:            46308 kB
Shmem:             29320 kB
Slab:             216512 kB
SReclaimable:     186456 kB
SUnreclaim:        30056 kB
KernelStack:        3792 kB
PageTables:         5748 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     536112 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:    104448 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       99712 kB
DirectMap2M:    33445888 kB

Free space on /tmp at the end= 59364 MiB
End job on node150 at 2019-09-16 00:57:24