Trace number 4408130

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
cosoco 2.0? (TO) 2519.82 2520.01

General information on the benchmark

NameQuasiGroups/QuasiGroups-elt-qg3/
QuasiGroup-3-12.xml
MD5SUMe29963f3eb1abf29691517033ab1fcf0
Bench CategoryCSP (decision problem)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark10.0419
Satisfiable
(Un)Satisfiability was proved
Number of variables288
Number of constraints266
Number of domains2
Minimum domain size12
Maximum domain size144
Distribution of domain sizes[{"size":12,"count":144},{"size":144,"count":132}]
Minimum variable degree0
Maximum variable degree135
Distribution of variable degrees[{"degree":0,"count":12},{"degree":2,"count":132},{"degree":134,"count":12},{"degree":135,"count":132}]
Minimum constraint arity3
Maximum constraint arity145
Distribution of constraint arities[{"arity":3,"count":132},{"arity":12,"count":1},{"arity":144,"count":1},{"arity":145,"count":132}]
Number of extensional constraints0
Number of intensional constraints132
Distribution of constraint types[{"type":"intension","count":132},{"type":"allDifferent","count":1},{"type":"instantiation","count":1},{"type":"element","count":132}]
Optimization problemNO
Type of objective

Solver Data 

0.00/0.03	c
0.00/0.03	c This is cosoco 2.00 --  
0.00/0.03	c
0.00/0.03	c command line: cosoco -lc -model -restarts -stick HOME/instance-4408130-1569801267.xml 
0.00/0.03	c intension  eq(add(mul(x[0][1],12),x[1][0]),y[0][1]) -> extension (support ? 1) - 144tuples
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c intension -> extension : same as previous
0.00/0.06	c inc root propagations     : 2620        
0.00/0.08	c restarts enabled, print stats at every restarts
0.00/0.08	c
0.00/0.08	c
0.00/0.08	tension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c intension -> extension : same as previous
0.00/0.08	c ========================================[ Problem Statistics ]===========================================
0.00/0.08	c |                                                                                                       
0.00/0.08	c |  Parse time        :         0.08 s 
0.00/0.08	c |
0.00/0.08	c |               Variables: 288
0.00/0.08	c |            Domain Sizes: 12..144
0.00/0.08	c |
0.00/0.08	c |             Constraints: 300
0.00/0.08	c |                   Arity: 1..145  -- Unary: 12  -- Ternary: 132
0.00/0.08	c | 
0.00/0.08	c |                   Types: 
0.00/0.08	c |                          All Different: 24
0.00/0.08	c |                          Element Constant: 132
0.00/0.08	c |                          Extension: 132  (nb tuples: 144..144) -- (shared: 131)
0.00/0.08	c |                          Unary: 12
0.00/0.08	c |                          
0.00/0.08	c =========================================================================================================
0.00/0.08	c conflicts      decisions      filterCalls    propagations   Root Prop.     Nogoods        min/max/avg depth
0.00/0.08	c remove useless variable y[0][0]
0.00/0.08	c remove useless variable y[1][1]
0.00/0.08	c remove useless variable y[2][2]
0.00/0.08	c remove useless variable y[3][3]
0.00/0.08	c remove useless variable y[4][4]
0.00/0.08	c remove useless variable y[5][5]
0.00/0.08	c remove useless variable y[6][6]
0.00/0.08	c remove useless variable y[7][7]
0.00/0.08	c remove useless variable y[8][8]
0.00/0.08	c remove useless variable y[9][9]
0.00/0.08	c remove useless variable y[10][10]
0.00/0.08	c remove useless variable y[11][11]
0.10/0.21	c 101            541            425075         181085         186            0              122   139   126
0.31/0.32	c 211            1019           784489         346644         186            0              20   89   76   
0.41/0.44	c 332            1467           1080663        465831         186            0              30   78   67   
0.60/0.62	c 465            2176           1706185        693253         186            0              18   88   74   
0.69/0.78	c 611            2895           2281978        926295         186            0              22   94   75   
0.89/0.98	c 771            3667           2837023        1180888        186            0              40   84   70   
1.19/1.20	c 947            4570           3632605        1521982        186            0              68   89   77   
1.39/1.43	c 1140           5501           4405314        1843311        186            0              60   81   71   
1.59/1.65	c 1352           6615           4997104        2053649        186            0              34   80   62   
1.89/1.90	c 1585           7465           5650698        2345955        186            0              18   78   64   
2.10/2.16	c 1841           8493           6323747        2623225        186            0              25   83   61   
2.49/2.55	c 2122           10225          7517835        3229885        186            0              24   94   71   
2.89/2.90	c 2431           11749          8761737        3783164        186            0              30   86   75   
3.29/3.34	c 2770           13152          10054256       4423133        186            0              32   82   67   
3.79/3.84	c 3142           15030          11678471       5218570        186            0              25   92   74   
4.19/4.30	c 3551           17023          13321728       5936190        186            0              68   94   78   
4.79/4.89	c 4000           19181          15229345       6735871        186            0              41   82   66   
5.49/5.59	c 4493           21684          17440253       7755307        186            0              32   89   67   
6.39/6.42	c 5035           24970          20085710       8949740        186            0              56   89   70   
7.00/7.06	c 5631           27671          22551931       9964841        186            0              30   95   80   
7.99/8.04	c 6286           31517          25605109       11468062       186            0              32   88   68   
8.79/8.83	c 7006           34621          28172348       12788291       186            0              45   86   75   
9.59/9.68	c 7798           38351          31131950       14135099       186            0              64   88   75   
10.79/10.82	c 8669           42869          34763103       15766110       186            0              57   90   69   
11.89/11.99	c 9627           47386          39010599       17716584       186            0              65   92   76   
13.19/13.23	c 10680          52380          43676689       19630650       186            0              55   91   77   
14.59/14.68	c 11838          59080          48738000       21909837       186            0              30   97   78   
15.89/15.98	c 13111          64599          53501204       23946439       186            0              67   91   79   
17.29/17.31	c 14511          70293          57935868       25660570       186            0              34   86   68   
18.89/18.99	c 16051          77026          62975134       27740751       186            0              20   85   66   
20.89/20.96	c 17745          85271          70179248       30540107       186            0              18   96   75   
23.49/23.56	c 19608          94061          78040943       34473565       186            0              18   91   70   
25.99/26.07	c 21657          103499         87044237       38543107       186            0              17   93   77   
29.09/29.13	c 23910          115599         97982261       43138818       186            0              38   93   74   
31.88/31.94	c 26388          127738         107407481      47028290       186            0              28   90   71   
34.78/34.81	c 29113          139447         118644766      51291005       186            0              68   97   77   
38.09/38.16	c 32110          152917         131279695      56157423       186            0              53   95   77   
42.38/42.44	c 35406          167801         143557661      62363338       186            0              27   91   69   
46.38/46.43	c 39031          185109         158348870      68427681       186            0              37   96   77   
50.88/50.91	c 43018          204284         175978072      75582918       186            0              29   94   77   
56.68/56.71	c 47403          225210         193415060      83695282       186            0              37   88   67   
62.38/62.41	c 52226          246234         213664882      92870670       186            0              37   90   75   
68.09/68.14	c 57531          269749         233781993      101184472      186            0              53   90   68   
74.38/74.49	c 63366          297614         259429303      111138835      186            0              23   99   78   
82.09/82.14	c 69784          330835         286434292      121868955      186            0              31   95   72   
91.99/92.05	c 76843          363646         317445504      137439928      186            0              51   93   70   
100.99/101.05	c 84607          401690         350997931      152083438      186            0              22   95   78   
110.38/110.49	c 93147          442292         384456184      168290883      186            0              49   96   76   
122.09/122.17	c 102541         492558         424829155      186299241      186            0              38   94   72   
134.58/134.60	c 112874         541301         465098870      204214375      186            0              39   87   69   
146.38/146.47	c 124240         594167         513690367      222342567      186            0              23   94   77   
158.98/159.07	c 136742         649823         551050120      237696014      186            0              18   90   68   
176.37/176.43	c 150494         725737         613581419      265117662      186            0              15   97   75   
193.47/193.53	c 165621         786348         664369331      288245576      186            0              42   87   67   
211.57/211.61	c 182260         869861         734800394      317701064      186            0              55   97   76   
232.17/232.28	c 200562         948646         800600395      344257343      186            0              40   91   66   
255.08/255.17	c 220694         1044575        883063301      377920386      186            0              53   97   72   
280.27/280.39	c 242839         1143004        968342409      417392889      186            0              51   94   74   
306.16/306.25	c 267198         1251178        1062917045     462315049      186            0              24   96   78   
342.66/342.72	c 293992         1384784        1178149675     519949026      186            0              43   95   70   
373.76/373.82	c 323465         1529830        1308954909     570805047      186            0              20   101   79  
412.26/412.30	c 355885         1688259        1441970910     630519129      186            0              29   93   72   
461.55/461.65	c 391547         1869519        1602251478     709948241      186            0              52   96   70   
505.65/505.77	c 430775         2054139        1772683200     779258384      186            0              37   96   77   
551.85/551.99	c 473925         2246992        1934477785     845490405      186            0              30   93   72   
617.34/617.45	c 521390         2452783        2134829461     949248677      186            0              31   94   68   
675.54/675.69	c 573601         2694183        2355976333     1041542708     186            0              37   99   77   
735.13/735.26	c 631033         2952449        2587646030     1138891824     186            0              23   100   77  
805.73/805.82	c 694208         3243985        2859541796     1252532519     186            0              27   99   78   
878.72/878.85	c 763700         3570851        3132228224     1363969470     186            0              26   96   74   
966.02/966.12	c 840141         3918020        3457682805     1501931620     186            0              39   95   76   
1065.51/1065.61	c 924226         4318230        3815647108     1642809285     186            0              52   94   70   
1187.40/1187.55	c 1016719        4733685        4201780794     1819874939     186            0              23   95   69   
1313.90/1314.10	c 1118461        5194338        4620835989     2014290032     186            0              27   95   69   
1436.38/1436.59	c 1230377        5705574        5045431972     2194694785     186            0              31   98   71   
1581.67/1581.80	c 1353484        6283854        5574225090     2424566368     186            0              36   99   76   
1750.86/1751.08	c 1488901        6918029        6146226635     2679602197     186            0              50   98   72   
1915.14/1915.32	c 1637859        7629500        6782082645     2925988608     186            0              26   98   75   
2127.53/2127.74	c 1801712        8455632        7493301038     3247491040     186            0              47   93   68   
2349.41/2349.65	c 1981950        9322755        8205694559     3567700982     186            0              38   97   68   
2519.80/2520.00	
2519.80/2520.00	c *** INTERRUPTED ***
2519.80/2520.00	c restarts              : 80          
2519.80/2520.00	c conflicts             : 2135703      (848 /sec)
2519.80/2520.00	c decisions             : 10028846     (3980 /sec)
2519.80/2520.00	c propagations          : 3835048500  (1521960 /sec)
2519.80/2520.00	c root propagations     : 2620        
2519.80/2520.00	c filter calls          : 8855812874     (3514477 /sec)
2519.80/2520.00	c useless filter calls  : 7596053314     (85 %)
2519.80/2520.00	c Memory used           : 29.93 MB
2519.80/2520.00	c CPU time              : 2519.81 s
2519.80/2520.00	c real time             : 2520 s
2519.80/2520.00	
2519.80/2520.00	s UNKNOWN

Verifier Data


Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (15227 MiB free)
  memory of node 1: 16384 MiB (15051 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4408130-1569801267/watcher-4408130-1569801267 -o /tmp/evaluation-result-4408130-1569801267/solver-4408130-1569801267 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node121-1569787350-1892 --watchdog 2580 cosoco -lc -model -restarts -stick HOME/instance-4408130-1569801267.xml 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=5323, runsolver pid=5320
Current StackSize limit: 8192 KiB


[startup+0.100819 s]*
/proc/loadavg: 2.46 2.15 2.09 4/201 5324
/proc/meminfo: memFree=31003300/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30516 memory=6976 CPUtime=0.1 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 2043 0 0 0 10 0 0 0 20 0 1 0 556389209 31248384 1744 33554432000 4194304 6211132 140728050161312 140728050158888 139815362026469 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7629 1744 861 493 0 934 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 30516 KiB
Current children cumulated memory: 6976 KiB

[startup+0.209629 s]*
/proc/loadavg: 2.46 2.15 2.09 3/201 5324
/proc/meminfo: memFree=31002928/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7240 CPUtime=0.21 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 2069 0 0 0 21 0 0 0 20 0 1 0 556389209 31383552 1810 33554432000 4194304 6211132 140728050161312 140728050158888 5099201 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1810 881 493 0 967 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7240 KiB

[startup+0.309273 s]*
/proc/loadavg: 2.46 2.15 2.09 3/201 5324
/proc/meminfo: memFree=31002928/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7240 CPUtime=0.31 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 2070 0 0 0 31 0 0 0 20 0 1 0 556389209 31383552 1810 33554432000 4194304 6211132 140728050161312 140728050158888 5099162 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1810 881 493 0 967 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7240 KiB

[startup+0.700332 s]
/proc/loadavg: 2.34 2.13 2.08 3/201 5324
/proc/meminfo: memFree=31002928/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7240 CPUtime=0.69 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 2070 0 0 0 69 0 0 0 20 0 1 0 556389209 31383552 1810 33554432000 4194304 6211132 140728050161312 140728050159272 5193728 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1810 881 493 0 967 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7240 KiB

[startup+1.50033 s]
/proc/loadavg: 2.34 2.13 2.08 3/201 5324
/proc/meminfo: memFree=31003248/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7240 CPUtime=1.49 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 2071 0 0 0 149 0 0 0 20 0 1 0 556389209 31383552 1810 33554432000 4194304 6211132 140728050161312 140728050159272 5038947 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1810 881 493 0 967 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7240 KiB

[startup+3.10925 s]
/proc/loadavg: 2.34 2.13 2.08 3/201 5324
/proc/meminfo: memFree=31003372/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=3.1 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 2948 0 0 0 310 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5099205 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+6.30033 s]
/proc/loadavg: 2.31 2.13 2.08 3/200 5324
/proc/meminfo: memFree=31002908/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=6.29 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 3827 0 0 0 629 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5193760 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+12.7003 s]
/proc/loadavg: 2.29 2.12 2.08 3/200 5324
/proc/meminfo: memFree=31003172/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=12.69 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 4705 0 0 0 1269 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5039199 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+25.5003 s]
/proc/loadavg: 2.24 2.12 2.08 3/200 5324
/proc/meminfo: memFree=31003544/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=25.49 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 6462 0 0 0 2549 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5099205 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 25.49 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+51.1092 s]
/proc/loadavg: 2.15 2.11 2.08 3/200 5327
/proc/meminfo: memFree=31003140/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=51.09 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 7343 0 0 0 5109 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5039092 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 51.09 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+102.3 s]
/proc/loadavg: 2.06 2.09 2.07 3/199 5332
/proc/meminfo: memFree=31002964/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=102.28 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 7715 0 0 0 10228 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5099124 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 102.28 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+162.3 s]
/proc/loadavg: 2.02 2.07 2.07 3/199 5336
/proc/meminfo: memFree=31003300/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=162.27 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 8086 0 0 0 16227 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5099205 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 162.27 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+222.3 s]
/proc/loadavg: 2.01 2.06 2.06 3/199 5340
/proc/meminfo: memFree=31003040/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=222.27 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 8457 0 0 0 22227 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5099205 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 222.27 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 2.05 2.06 3/199 5347
/proc/meminfo: memFree=31002572/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=282.27 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 8828 0 0 0 28227 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5193760 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 282.27 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.04 2.05 3/199 5351
/proc/meminfo: memFree=31002648/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=342.26 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 9198 0 0 0 34226 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5193728 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 342.26 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.04 2.05 3/200 5363
/proc/meminfo: memFree=31002472/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=402.26 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 9569 0 0 0 40226 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5099190 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 402.26 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/201 5395
/proc/meminfo: memFree=31001340/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=462.25 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 9939 0 0 0 46225 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5397511 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 462.25 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB


################
# More data... #
################

/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 1722.16 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+1782.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 5576
/proc/meminfo: memFree=31002036/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=1782.15 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 17902 0 0 0 178215 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5193760 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 1782.15 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+1842.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 5580
/proc/meminfo: memFree=31001788/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=1842.15 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 18248 0 0 0 184215 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5079308 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 1842.15 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+1902.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 5587
/proc/meminfo: memFree=31002632/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=1902.14 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 18594 0 0 0 190214 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5390587 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 1902.14 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+1962.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 5591
/proc/meminfo: memFree=31002176/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=1962.14 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 18966 0 0 0 196214 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5099197 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 1962.14 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+2022.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 5595
/proc/meminfo: memFree=31001632/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=2022.13 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 19314 0 0 0 202213 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5099205 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 2022.13 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+2082.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 5602
/proc/meminfo: memFree=31001544/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=2082.13 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 19662 0 0 0 208213 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5390177 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 2082.13 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+2142.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 5605
/proc/meminfo: memFree=31001984/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=2142.13 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 20032 0 0 0 214213 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5099313 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 2142.13 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+2202.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 5616
/proc/meminfo: memFree=31001984/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=2202.12 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 20382 0 0 0 220212 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5099197 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 2202.12 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+2262.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 5622
/proc/meminfo: memFree=31001296/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=2262.12 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 20732 0 0 0 226212 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5099155 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 2262.12 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+2322.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 5626
/proc/meminfo: memFree=31001156/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=2322.11 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 21082 0 0 0 232211 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5393095 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 2322.11 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+2382.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 5630
/proc/meminfo: memFree=31001924/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=2382.11 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 21452 0 0 0 238211 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5037823 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 2382.11 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+2442.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 5636
/proc/meminfo: memFree=31001828/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=2442.1 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 21802 0 0 0 244210 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5099229 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 2442.1 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

[startup+2502.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 5640
/proc/meminfo: memFree=31001748/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=2502.1 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 22153 0 0 0 250210 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5099197 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 2502.1 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+2520 s]*
/proc/loadavg: 2.00 2.01 2.05 3/200 5643
/proc/meminfo: memFree=31001964/32770624 swapFree=66993544/67108860
[pid=5323] ppid=5320 vsize=30648 memory=7312 CPUtime=2519.8 cores=0,2,4,6
/proc/5323/stat : 5323 (cosoco) R 5320 5323 1861 0 -1 4202496 22504 0 0 0 251980 0 0 0 20 0 1 0 556389209 31383552 1828 33554432000 4194304 6211132 140728050161312 140728050159272 5039432 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 32505856 140728050164747 140728050164849 140728050164849 140728050167758 0
/proc/5323/statm: 7662 1828 881 493 0 967 0
Current children cumulated CPU time: 2519.8 s
Current children cumulated vsize: 30648 KiB
Current children cumulated memory: 7312 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

[startup+2520.01 s]
# the end of solver process 5323 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2519.81 s, system=0.009999 s

Solver just ended.
??? end of watcher thread
Dumping a history of the last processes samples

Child status: 1
Real time (s): 2520.01
CPU time (s): 2519.82
CPU user time (s): 2519.81
CPU system time (s): 0.009999
CPU usage (%): 99.9924
Max. virtual memory (cumulated for all children) (KiB): 30648
Max. memory (cumulated for all children) (KiB): 7312

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2519.81
system time used= 0.009999
maximum resident set size= 7376
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 22553
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 15
involuntary context switches= 3158


# summary of solver processes directly reported to runsolver:
#   pid: 5323
#   total CPU time (s): 2519.82
#   total CPU user time (s): 2519.81
#   total CPU system time (s): 0.009999

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 9.732 second user time and 15.8716 second system time

The end

Launcher Data 

Begin job on node121 at 2019-09-30 01:53:43
IDJOB=4408130
IDBENCH=133813
IDSOLVER=2874
FILE ID=node121/4408130-1569801267
RUNJOBID= node121-1569787350-1892
PBS_JOBID= 21705831
Free space on /tmp= 62028 MiB

SOLVER NAME= cosoco 2.0
BENCH NAME= XCSP17/QuasiGroups/QuasiGroups-elt-qg3/QuasiGroup-3-12.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4408130-1569801267/watcher-4408130-1569801267 -o /tmp/evaluation-result-4408130-1569801267/solver-4408130-1569801267 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node121-1569787350-1892 --watchdog 2580  cosoco -lc -model -restarts -stick HOME/instance-4408130-1569801267.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= e29963f3eb1abf29691517033ab1fcf0
RANDOM SEED=1262183509

node121.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        31005948 kB
MemAvailable:   31866452 kB
Buffers:          206964 kB
Cached:           977020 kB
SwapCached:         4540 kB
Active:           680976 kB
Inactive:         513924 kB
Active(anon):       4808 kB
Inactive(anon):    80912 kB
Active(file):     676168 kB
Inactive(file):   433012 kB
Unevictable:      109048 kB
Mlocked:          109048 kB
SwapTotal:      67108860 kB
SwapFree:       66993544 kB
Dirty:              2748 kB
Writeback:             0 kB
AnonPages:        116096 kB
Mapped:            28580 kB
Shmem:             66188 kB
Slab:             163660 kB
SReclaimable:     134316 kB
SUnreclaim:        29344 kB
KernelStack:        3536 kB
PageTables:         5120 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     570820 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     67584 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      122240 kB
DirectMap2M:    33423360 kB

Free space on /tmp at the end= 62024 MiB
End job on node121 at 2019-09-30 02:35:44