Trace number 4267341

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
cosoco-mini 1.12? (TO) 2400.03 2400.01

General information on the benchmark

NameCoveringArray/CoveringArray-elt-s1/
CoveringArray-elt-3-09-2-12.xml
MD5SUM184e4a42ac40b5c8b3b859f3d89928ae
Bench CategoryCSP (decision problem)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1680
Number of constraints768
Number of domains2
Minimum domain size8
Maximum domain size12
Distribution of domain sizes[{"size":8,"count":1008},{"size":12,"count":672}]
Minimum variable degree2
Maximum variable degree9
Distribution of variable degrees[{"degree":2,"count":672},{"degree":9,"count":1008}]
Minimum constraint arity8
Maximum constraint arity84
Distribution of constraint arities[{"arity":8,"count":84},{"arity":13,"count":672},{"arity":84,"count":12}]
Number of extensional constraints12
Number of intensional constraints0
Distribution of constraint types[{"type":"extension","count":12},{"type":"allDifferent","count":84},{"type":"element","count":672}]
Optimization problemNO
Type of objective

Solver Data 

0.00/0.07	c root propagations     : 0           
0.00/0.07	c restarts enabled, print stats at every restarts
0.00/0.07	c
0.00/0.07	c
0.00/0.07	c
0.00/0.07	c This is cosoco 1.12 --  
0.00/0.07	c
0.00/0.07	c command line: cosoco -lc -model -restarts HOME/instance-4267341-1502615590.xml 
0.00/0.07	c ========================================[ Problem Statistics ]===========================================
0.00/0.07	c |                                                                                                       
0.00/0.07	c |  Parse time        :         0.06 s 
0.00/0.07	c |
0.00/0.07	c |               Variables: 1680
0.00/0.07	c |            Domain Sizes: 8..12
0.00/0.07	c |
0.00/0.07	c |             Constraints: 768
0.00/0.07	c |                   Arity: 8..84
0.00/0.07	c | 
0.00/0.07	c |                   Types: 
0.00/0.07	c |                          All Different: 84
0.00/0.07	c |                          Element Constant: 672
0.00/0.07	c |                          Extension: 12  (nb tuples: 512..512) -- (shared: 11)
0.00/0.07	c |                          
0.00/0.07	c =========================================================================================================
0.00/0.07	c conflicts      decisions      filterCalls    propagations   Root Prop.     Nogoods        min/max/avg depth
0.10/0.15	c 101            862            124581         122172         0              0              628   639   626
0.20/0.28	c 211            1710           212806         311399         0              0              28   95   78   
0.30/0.34	c 332            2051           270308         372141         0              0              35   53   41   
0.40/0.46	c 465            3605           381968         580017         0              0              48   232   104 
0.60/0.62	c 611            4164           505200         873698         0              0              59   76   68   
0.69/0.74	c 771            4677           628486         1045568        0              0              116   128   120
0.79/0.85	c 947            5170           720649         1197364        0              0              50   78   66   
0.89/1.00	c 1140           5688           825975         1409802        0              0              55   73   63   
1.10/1.15	c 1352           6362           1002461        1611317        0              0              66   73   69   
1.29/1.32	c 1585           8168           1139881        1841877        0              0              80   115   105 
1.49/1.54	c 1841           11542          1290071        2146934        0              0              88   153   122 
1.69/1.76	c 2122           12831          1465081        2492897        0              0              70   97   83   
1.99/2.03	c 2431           14454          1665709        2934014        0              0              65   106   84  
2.29/2.34	c 2770           15970          1922814        3433782        0              0              68   96   79   
2.69/2.74	c 3142           17513          2219168        4079413        0              0              74   102   86  
3.10/3.10	c 3551           19226          2463684        4664267        0              0              80   111   92  
3.49/3.53	c 4000           20919          2835075        5366299        0              0              91   115   100 
3.79/3.87	c 4493           22725          3114612        5831373        0              0              74   108   87  
4.29/4.30	c 5035           24313          3582892        6536461        0              0              70   84   75   
4.59/4.67	c 5631           26212          3909687        7045528        0              0              73   103   85  
4.89/4.92	c 6286           27556          4179791        7338021        0              0              71   93   78   
5.39/5.47	c 7006           40160          4546181        8100073        0              0              68   176   121 
5.99/6.07	c 7798           43854          5045695        9196899        0              0              68   98   86   
6.89/6.90	c 8669           47489          5697266        10634921       0              0              76   100   90  
7.79/7.86	c 9627           51840          6465054        12343058       0              0              83   108   96  
8.29/8.35	c 10680          55400          7067867        13057322       0              0              84   113   95  
8.89/8.94	c 11838          59269          7629064        13903900       0              0              73   102   89  
9.39/9.46	c 13111          62182          8267312        14508249       0              0              72   89   81   
10.69/10.75	c 14511          67477          9260251        16832676       0              0              56   95   74   
11.89/11.90	c 16051          72869          10311074       18767866       0              0              61   98   79   
12.99/13.05	c 17745          78050          11296643       20615004       0              0              74   98   84   
14.69/14.71	c 19608          83995          12904813       23550622       0              0              60   89   73   
15.99/16.05	c 21657          90614          14053994       25676548       0              0              70   98   86   
16.89/16.91	c 23910          95084          15025549       26616935       0              0              68   87   77   
19.49/19.50	c 26388          100909         16798248       30922692       0              0              61   76   66   
21.99/22.05	c 29113          107043         18623971       35016060       0              0              65   86   73   
24.58/24.70	c 32110          116841         21010271       39425247       0              0              67   99   81   
26.38/26.49	c 35406          123989         22626856       41933673       0              0              67   94   81   
29.78/29.89	c 39031          134826         25624529       48026006       0              0              64   91   76   
31.88/31.95	c 43018          144302         27676269       50959314       0              0              66   95   76   
34.58/34.61	c 47403          208306         29789899       55481008       0              0              58   195   145 
39.09/39.11	c 52226          222399         33385833       63107222       0              0              64   102   77  
43.18/43.23	c 57531          240217         37127755       69559856       0              0              75   102   87  
46.88/46.92	c 63366          334901         40102222       74820594       0              0              77   189   148 
52.38/52.45	c 69784          360496         44651943       84894534       0              0              69   107   90  
55.68/55.75	c 76843          377352         48265780       89324545       0              0              70   99   82   
62.88/62.92	c 84607          400416         53600146       101622641      0              0              64   101   77  
70.09/70.19	c 93147          425237         59161614       113103471      0              0              68   104   83  
76.59/76.60	c 102541         462941         65458655       123179735      0              0              76   106   93  
81.79/81.81	c 112874         490346         70764299       130249072      0              0              75   102   85  
91.19/91.20	c 124240         526571         78837611       145805651      0              0              69   96   80   
101.19/101.26	c 136742         569013         86863946       162955842      0              0              68   102   85  
111.69/111.78	c 150494         620248         96644654       182091289      0              0              66   104   84  
120.49/120.53	c 165621         670014         105158575      195529621      0              0              70   106   89  
133.99/134.01	c 182260         726095         118585113      219991766      0              0              66   103   83  
146.69/146.74	c 200562         783048         129364355      241661563      0              0              65   102   85  
165.69/165.70	c 220694         850794         146190228      278123390      0              0              67   101   81  
179.39/179.41	c 242839         913778         159612147      300339562      0              0              63   98   76   
195.39/195.42	c 267198         999680         174504657      327439431      0              0              65   99   81   
216.89/216.99	c 293992         1093081        193038686      366315111      0              0              67   100   85  
233.19/233.25	c 323465         1176965        210708619      391746399      0              0              64   100   78  
256.10/256.17	c 355885         1301579        233087422      431102215      0              0              62   101   81  
282.19/282.23	c 391547         1426456        257164939      477521387      0              0              62   186   90  
311.39/311.47	c 430775         1535261        279628826      525395054      0              0              66   101   85  
336.99/337.03	c 473925         1825321        304787939      571529459      0              0              63   183   119 
366.59/366.67	c 521390         1967407        332089608      619906648      0              0              65   104   83  
406.69/406.78	c 573601         2223503        367138054      690616145      0              0              62   166   89  
446.69/446.77	c 631033         2440201        395851850      760328441      0              0              64   117   95  
493.79/493.82	c 694208         2662178        434689357      837495824      0              0              65   106   87  
541.19/541.24	c 763700         2947705        480068313      917447385      0              0              67   108   92  
585.19/585.25	c 840141         3198187        521330133      984538257      0              0              63   107   86  
642.19/642.26	c 924226         3563289        574467795      1080438942     0              0              64   107   91  
708.69/708.75	c 1016719        4078698        633783146      1198841823     0              0              62   177   102 
769.89/769.94	c 1118461        4453657        692758294      1294159696     0              0              65   108   89  
863.29/863.33	c 1230377        4932717        775899726      1462553835     0              0              63   105   87  
926.69/926.71	c 1353484        5267924        838297234      1555666375     0              0              63   103   82  
1030.00/1030.06	c 1488901        5843623        934649010      1733124192     0              0              62   105   86  
1119.60/1119.63	c 1637859        7409763        1009375836     1885767487     0              0              65   181   135 
1271.30/1271.32	c 1801712        8112921        1143303225     2164417297     0              0              67   114   93  
1377.60/1377.68	c 1981950        8804701        1246227615     2329343295     0              0              63   106   88  
1510.40/1510.43	c 2180211        10656373       1368162828     2554460632     0              0              60   170   117 
1641.20/1641.22	c 2398298        11428738       1490448626     2762363245     0              0              64   109   86  
1827.60/1827.66	c 2638193        12446746       1663882704     3089889429     0              0              62   107   88  
2083.02/2083.05	c 2902077        13575360       1878576132     3566534795     0              0              65   113   91  
2276.61/2276.67	c 3192349        14967855       2050114772     3890377439     0              0              63   168   97  
2400.02/2400.00	
2400.02/2400.00	c *** INTERRUPTED ***
2400.02/2400.00	c restarts              : 85          
2400.02/2400.00	c conflicts             : 3353402      (1397 /sec)
2400.02/2400.00	c decisions             : 15743974     (6560 /sec)
2400.02/2400.00	c propagations          : 4111638656  (1713170 /sec)
2400.02/2400.00	c root propagations     : 0           
2400.02/2400.00	c filter calls          : 2155652851     (898182 /sec)
2400.02/2400.00	c useless filter calls  : 1188540167     (55 %)
2400.02/2400.00	c Memory used           : 37.79 MB
2400.02/2400.00	c CPU time              : 2400.02 s
2400.02/2400.00	
2400.02/2400.00	s UNKNOWN

Verifier Data


Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (15338 MiB free)
  memory of node 1: 16384 MiB (15641 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4267341-1502615590/watcher-4267341-1502615590 -o /tmp/evaluation-result-4267341-1502615590/solver-4267341-1502615590 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node103-1502602184-31005 --watchdog 2560 cosoco -lc -model -restarts HOME/instance-4267341-1502615590.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2500 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=1071, runsolver pid=1068
Current StackSize limit: 8192 KiB


[startup+0.100162 s]*
/proc/loadavg: 2.00 2.01 2.05 3/187 1072
/proc/meminfo: memFree=31710608/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15212 CPUtime=0.1 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 3990 0 0 0 10 0 0 0 20 0 1 0 94647203 39620608 3803 33554432000 4194304 5748600 140723395020560 140723395017864 4920135 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3803 763 380 0 3091 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15212 KiB

[startup+0.209639 s]*
/proc/loadavg: 2.00 2.01 2.05 3/187 1072
/proc/meminfo: memFree=31710608/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15212 CPUtime=0.2 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 3993 0 0 0 20 0 0 0 20 0 1 0 94647203 39620608 3803 33554432000 4194304 5748600 140723395020560 140723395017864 4995528 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3803 763 380 0 3091 0
Current children cumulated CPU time: 0.2 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15212 KiB

[startup+0.309033 s]*
/proc/loadavg: 2.00 2.01 2.05 3/187 1072
/proc/meminfo: memFree=31710608/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15212 CPUtime=0.3 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 3993 0 0 0 30 0 0 0 20 0 1 0 94647203 39620608 3803 33554432000 4194304 5748600 140723395020560 140723395017864 4894412 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3803 763 380 0 3091 0
Current children cumulated CPU time: 0.3 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15212 KiB

[startup+0.70028 s]
/proc/loadavg: 2.00 2.01 2.05 3/187 1072
/proc/meminfo: memFree=31710608/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15212 CPUtime=0.69 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 3993 0 0 0 69 0 0 0 20 0 1 0 94647203 39620608 3803 33554432000 4194304 5748600 140723395020560 140723395017864 4894405 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3803 763 380 0 3091 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15212 KiB

[startup+1.50027 s]
/proc/loadavg: 2.00 2.01 2.05 3/187 1072
/proc/meminfo: memFree=31710608/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15212 CPUtime=1.49 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 3993 0 0 0 149 0 0 0 20 0 1 0 94647203 39620608 3803 33554432000 4194304 5748600 140723395020560 140723395017864 5122362 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3803 763 380 0 3091 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15212 KiB

[startup+3.10906 s]
/proc/loadavg: 1.92 2.00 2.04 3/187 1072
/proc/meminfo: memFree=31710656/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=3.1 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 6359 0 0 0 310 0 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 4894412 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+6.30027 s]
/proc/loadavg: 1.92 2.00 2.04 3/187 1072
/proc/meminfo: memFree=31710384/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=6.29 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 8389 0 0 0 629 0 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 4894309 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+12.7003 s]
/proc/loadavg: 1.93 2.00 2.04 3/187 1072
/proc/meminfo: memFree=31710520/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=12.69 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 8819 0 0 0 1269 0 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 4894412 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+25.5003 s]
/proc/loadavg: 1.95 2.00 2.04 3/187 1072
/proc/meminfo: memFree=31710520/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=25.48 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 9675 0 0 0 2548 0 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 4915636 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+51.109 s]
/proc/loadavg: 1.96 2.00 2.04 3/187 1072
/proc/meminfo: memFree=31710520/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=51.09 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 10109 0 0 0 5109 0 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 4918957 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 51.09 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+102.3 s]
/proc/loadavg: 1.98 2.00 2.04 3/187 1080
/proc/meminfo: memFree=31709232/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=102.29 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 10538 0 0 0 10229 0 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 4918903 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 102.29 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+162.3 s]
/proc/loadavg: 1.99 2.00 2.04 3/188 1112
/proc/meminfo: memFree=31709208/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=162.29 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 10967 0 0 0 16229 0 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 5122362 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 162.29 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+222.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/188 1113
/proc/meminfo: memFree=31709588/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=222.29 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 11396 0 0 0 22229 0 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 4894279 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 222.29 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/188 1114
/proc/meminfo: memFree=31709200/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=282.29 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 11827 0 0 0 28229 0 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 4894302 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 282.29 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/188 1115
/proc/meminfo: memFree=31709780/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=342.29 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 12257 0 0 0 34229 0 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 4894318 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 342.29 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/188 1116
/proc/meminfo: memFree=31709640/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=402.29 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 12687 0 0 0 40229 0 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 4890592 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 402.29 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/187 1117
/proc/meminfo: memFree=31710012/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=462.29 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 13116 0 0 0 46229 0 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 4894405 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 462.29 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB


################
# More data... #
################

/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 1602.3 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+1662.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/187 1221
/proc/meminfo: memFree=31691804/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=1662.3 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 21330 0 0 0 166230 0 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 5126832 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 1662.3 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+1722.3 s]
/proc/loadavg: 2.34 2.08 2.07 3/187 1222
/proc/meminfo: memFree=31691908/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=1722.3 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 21706 0 0 0 172230 0 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 4894272 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 1722.3 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+1782.3 s]
/proc/loadavg: 2.23 2.11 2.08 3/187 1224
/proc/meminfo: memFree=31691852/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=1782.3 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 22094 0 0 0 178230 0 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 4894318 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 1782.3 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+1842.3 s]
/proc/loadavg: 2.08 2.09 2.07 3/187 1225
/proc/meminfo: memFree=31691792/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=1842.3 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 22524 0 0 0 184230 0 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 4894829 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 1842.3 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+1902.3 s]
/proc/loadavg: 2.03 2.07 2.06 3/188 1234
/proc/meminfo: memFree=31691480/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=1902.31 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 22897 0 0 0 190230 1 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 4920181 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 1902.31 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+1962.3 s]
/proc/loadavg: 2.01 2.06 2.06 3/188 1235
/proc/meminfo: memFree=31691604/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=1962.31 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 23267 0 0 0 196230 1 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 5122352 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 1962.31 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+2022.3 s]
/proc/loadavg: 2.00 2.05 2.05 3/188 1236
/proc/meminfo: memFree=31691812/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=2022.31 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 23646 0 0 0 202230 1 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 4920173 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 2022.31 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+2082.3 s]
/proc/loadavg: 2.00 2.04 2.05 3/188 1237
/proc/meminfo: memFree=31691756/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=2082.31 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 24017 0 0 0 208230 1 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 4893138 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 2082.31 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+2142.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/188 1238
/proc/meminfo: memFree=31691944/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=2142.31 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 24452 0 0 0 214230 1 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 4894306 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 2142.31 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+2202.3 s]
/proc/loadavg: 2.08 2.04 2.05 3/187 1239
/proc/meminfo: memFree=31691900/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=2202.31 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 24828 0 0 0 220230 1 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 5144157 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 2202.31 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+2262.3 s]
/proc/loadavg: 2.03 2.04 2.05 3/186 1240
/proc/meminfo: memFree=31692728/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=2262.31 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 25204 0 0 0 226230 1 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 4894306 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 2262.31 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+2322.3 s]
/proc/loadavg: 2.01 2.03 2.05 3/187 1242
/proc/meminfo: memFree=31691324/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=2322.31 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 25660 0 0 0 232230 1 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 4894294 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 2322.31 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

[startup+2382.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/187 1243
/proc/meminfo: memFree=31691512/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=2382.31 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 26036 0 0 0 238230 1 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 5000320 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 2382.31 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+2400 s]*
/proc/loadavg: 2.00 2.02 2.05 3/187 1243
/proc/meminfo: memFree=31691512/32770624 swapFree=67108860/67108860
[pid=1071] ppid=1068 vsize=38692 memory=15324 CPUtime=2400.02 cores=1,3,5,7
/proc/1071/stat : 1071 (cosoco) R 1068 1071 30973 0 -1 4202496 26402 0 0 0 240001 1 0 0 20 0 1 0 94647203 39620608 3831 33554432000 4194304 5748600 140723395020560 140723395017960 4894479 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 7849824 7854664 12066816 140723395028159 140723395028254 140723395028254 140723395030990 0
/proc/1071/statm: 9673 3831 765 380 0 3091 0
Current children cumulated CPU time: 2400.02 s
Current children cumulated vsize: 38692 KiB
Current children cumulated memory: 15324 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

[startup+2400.01 s]
# the end of solver process 1071 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2400.02 s, system=0.01 s

Solver just ended.
??? end of watcher thread
Dumping a history of the last processes samples

Child status: 1
Real time (s): 2400.01
CPU time (s): 2400.03
CPU user time (s): 2400.02
CPU system time (s): 0.01
CPU usage (%): 100.001
Max. virtual memory (cumulated for all children) (KiB): 38692
Max. memory (cumulated for all children) (KiB): 15324

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2400.02
system time used= 0.01
maximum resident set size= 15396
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 26453
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 15
involuntary context switches= 671


# summary of solver processes directly reported to runsolver:
#   pid: 1071
#   total CPU time (s): 2400.03
#   total CPU user time (s): 2400.02
#   total CPU system time (s): 0.01

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 8.09219 second user time and 15.4368 second system time

The end

Launcher Data 

Begin job on node103 at 2017-08-13 10:58:51
IDJOB=4267341
IDBENCH=139771
IDSOLVER=2657
FILE ID=node103/4267341-1502615590
RUNJOBID= node103-1502602184-31005
PBS_JOBID= 20628296
Free space on /tmp= 61792 MiB

SOLVER NAME= cosoco-mini 1.12
BENCH NAME= XCSP17/CoveringArray/CoveringArray-elt-s1/CoveringArray-elt-3-09-2-12.xml
COMMAND LINE= cosoco -lc -model -restarts BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4267341-1502615590/watcher-4267341-1502615590 -o /tmp/evaluation-result-4267341-1502615590/solver-4267341-1502615590 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node103-1502602184-31005 --watchdog 2560  cosoco -lc -model -restarts HOME/instance-4267341-1502615590.xml

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 184e4a42ac40b5c8b3b859f3d89928ae
RANDOM SEED=1515314532

node103.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        31723484 kB
MemAvailable:   31994812 kB
Buffers:           88544 kB
Cached:           477336 kB
SwapCached:            0 kB
Active:           490244 kB
Inactive:         131888 kB
Active(anon):       5188 kB
Inactive(anon):    78704 kB
Active(file):     485056 kB
Inactive(file):    53184 kB
Unevictable:        6412 kB
Mlocked:            6412 kB
SwapTotal:      67108860 kB
SwapFree:       67108860 kB
Dirty:              1964 kB
Writeback:             0 kB
AnonPages:         53640 kB
Mapped:            44544 kB
Shmem:             25228 kB
Slab:             123848 kB
SReclaimable:      97848 kB
SUnreclaim:        26000 kB
KernelStack:        3360 kB
PageTables:         5328 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     351792 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      114048 kB
DirectMap2M:    33431552 kB

Free space on /tmp at the end= 61788 MiB
End job on node103 at 2017-08-13 11:38:52