Trace number 4243121

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
R1: cosoco 1.1? (TO) 2400.08 2400.31

General information on the benchmark

NameLatinSquare/LatinSquare-m1-gp/
qwh-o30-h375-17.xml
MD5SUM946d300933e85b0b5e5d11b6e9e445b0
Bench CategoryCSP (decision problem)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark5.0790401
Satisfiable
(Un)Satisfiability was proved
Number of variables900
Number of constraints2
Number of domains1
Minimum domain size30
Maximum domain size30
Distribution of domain sizes[{"size":30,"count":900}]
Minimum variable degree1
Maximum variable degree2
Distribution of variable degrees[{"degree":1,"count":375},{"degree":2,"count":525}]
Minimum constraint arity525
Maximum constraint arity900
Distribution of constraint arities[{"arity":525,"count":1},{"arity":900,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"allDifferent","count":1},{"type":"instantiation","count":1}]
Optimization problemNO
Type of objective

Solver Data 

0.00/0.05	c root propagations     : 4831        
0.00/0.05	c restarts enabled, print stats at every restarts
0.00/0.05	c
0.00/0.05	c
0.00/0.05	c
0.00/0.05	c This is cosoco 1.0 --  
0.00/0.05	c
0.00/0.05	c ========================================[ Problem Statistics ]===========================================
0.00/0.05	c |                                                                                                       
0.00/0.05	c |  Parse time        :         0.01 s 
0.00/0.05	c |
0.00/0.05	c |               Variables: 900
0.00/0.05	c |            Domain Sizes: 30..30
0.00/0.05	c |
0.00/0.05	c |             Constraints: 585
0.00/0.05	c |                   Arity: 1..30  -- Unary: 525
0.00/0.05	c | 
0.00/0.05	c |                   Types: 
0.00/0.05	c |                          All Different: 60
0.00/0.05	c |                          Unary: 525
0.00/0.05	c |                          
0.00/0.05	c =========================================================================================================
0.00/0.05	c conflicts      decisions      filterCalls    propagations   Root Prop.     Nogoods        min/max/avg depth
0.09/0.13	c 101            1049           14366          45541          898            0              590   660   641
0.19/0.21	c 211            2151           28245          106041         898            0              415   688   657
0.19/0.28	c 332            3483           41846          165452         898            0              498   689   643
0.29/0.35	c 465            4823           55017          214352         898            0              546   675   609
0.39/0.46	c 611            6502           75259          307812         898            0              577   716   669
0.49/0.52	c 771            7857           86412          345167         898            0              595   677   656
0.59/0.61	c 947            9249           103198         418305         898            0              557   670   622
0.69/0.75	c 1140           10851          127658         538015         898            0              560   714   673
0.79/0.86	c 1352           12778          148533         631344         898            0              534   720   682
0.89/0.97	c 1585           14503          168890         720441         898            0              573   701   668
1.10/1.12	c 1841           16657          197474         840094         898            0              579   721   662
1.20/1.25	c 2122           18682          222018         944433         898            0              559   709   651
1.29/1.38	c 2431           20590          244784         1032245        898            0              550   608   594
1.49/1.58	c 2770           22896          281919         1223934        898            0              565   728   684
1.69/1.79	c 3142           25827          319307         1422063        898            0              575   730   705
1.99/2.07	c 3551           28380          370355         1694378        898            0              618   716   686
2.19/2.24	c 4000           31226          402453         1845681        898            0              585   710   674
2.49/2.56	c 4493           34833          460801         2144464        898            0              652   722   695
2.69/2.79	c 5035           38536          503072         2329745        898            0              608   707   675
3.09/3.12	c 5631           42540          563980         2628440        898            0              594   763   714
3.39/3.46	c 6286           47101          626834         2923410        898            0              601   724   683
3.79/3.83	c 7006           51070          694075         3235530        898            0              575   714   675
4.09/4.16	c 7798           54435          752518         3514686        898            0              552   683   619
4.49/4.59	c 8669           59634          831596         3886261        898            0              552   748   695
4.99/5.09	c 9627           64260          922497         4319384        898            0              572   736   682
5.49/5.54	c 10680          69630          1005101        4701496        898            0              548   657   621
5.89/5.99	c 11838          75065          1087335        5109211        898            0              571   733   696
6.49/6.57	c 13111          83095          1194819        5575362        898            0              579   717   672
7.19/7.29	c 14511          90414          1324347        6252593        898            0              595   734   701
7.89/7.98	c 16051          98301          1450905        7022929        898            0              578   744   709
8.69/8.77	c 17745          106929         1595670        7759400        898            0              606   730   685
9.69/9.79	c 19608          119100         1780210        8678601        898            0              561   727   690
10.79/10.87	c 21657          129533         1976073        9666359        898            0              565   729   696
12.29/12.38	c 23910          143282         2247150        11279788       898            0              573   749   705
13.39/13.42	c 26388          153486         2431540        12281258       898            0              564   680   657
14.29/14.40	c 29113          164416         2612658        13167720       898            0              596   730   699
16.18/16.20	c 32110          178343         2935689        14835095       898            0              584   713   685
17.59/17.64	c 35406          193202         3196728        16148242       898            0              587   723   684
19.28/19.36	c 39031          213535         3516778        17552915       898            0              629   734   693
21.08/21.12	c 43018          230393         3836169        19264885       898            0              568   722   674
23.38/23.42	c 47403          253893         4251323        21358675       898            0              584   731   664
25.78/25.85	c 52226          274820         4690257        23638318       898            0              546   721   686
28.48/28.54	c 57531          298502         5178963        26425311       898            0              549   754   717
30.68/30.77	c 63366          320544         5588207        28422859       898            0              603   706   671
34.28/34.34	c 69784          350321         6238624        31678912       898            0              565   736   688
37.18/37.28	c 76843          375608         6774145        34495478       898            0              557   722   683
40.58/40.70	c 84607          407664         7397424        37640467       898            0              555   711   674
44.78/44.85	c 93147          448746         8151544        41154998       898            0              555   722   653
49.88/49.95	c 102541         495420         9082926        45935473       898            0              561   752   692
54.88/54.94	c 112874         542020         9981302        50906378       898            0              561   745   708
61.28/61.35	c 124240         590836         11138556       56980793       898            0              561   731   695
67.38/67.41	c 136742         658876         12256676       62510359       898            0              564   722   686
73.98/74.06	c 150494         716382         13457450       68643636       898            0              575   732   699
82.88/82.92	c 165621         786382         15047536       76629376       898            0              595   746   688
90.88/90.98	c 182260         859649         16504313       84493780       898            0              591   729   680
101.18/101.30	c 200562         962305         18383143       94087763       898            0              616   756   700
112.87/112.98	c 220694         1059569        20496247       104621658      898            0              585   736   695
123.57/123.61	c 242839         1153252        22424892       115364147      898            0              588   754   703
133.77/133.83	c 267198         1272793        24313392       125556580      898            0              571   742   699
146.67/146.77	c 293992         1391188        26672263       136935494      898            0              583   715   667
162.67/162.78	c 323465         1540075        29574700       151836613      898            0              587   726   682
176.87/176.93	c 355885         1690120        32158116       164576820      898            0              611   718   677
195.47/195.57	c 391547         1871912        35557885       181243385      898            0              564   733   688
216.77/216.81	c 430775         2047849        39408272       201919525      898            0              572   732   690
240.56/240.64	c 473925         2248501        43700287       222707172      898            0              569   726   674
266.17/266.23	c 521390         2485033        48319959       245863166      898            0              571   735   671
288.76/288.82	c 573601         2729053        52408661       266131664      898            0              555   722   675
323.26/323.31	c 631033         3062316        58671362       299625592      898            0              554   764   703
358.35/358.45	c 694208         3371659        65020225       333488335      898            0              602   745   690
395.35/395.41	c 763700         3702792        71748080       367018946      898            0              558   735   682
438.64/438.70	c 840141         4140566        79679991       403292912      898            0              558   745   679
481.85/481.98	c 924226         4541832        87527100       443461084      898            0              556   745   703
533.84/533.99	c 1016719        5091384        96991442       488821747      898            0              556   756   694
572.43/572.53	c 1118461        5460563        104023968      526675297      898            0              578   743   694
631.33/631.49	c 1230377        5939414        114677873      583230869      898            0              571   729   688
688.32/688.46	c 1353484        6505384        125034870      637049036      898            0              588   750   697
755.32/755.40	c 1488901        7010095        137023694      704185950      898            0              585   718   670
832.72/832.88	c 1637859        7695375        151052840      779519465      898            0              586   752   698
899.71/899.82	c 1801712        8365237        163257656      837972447      898            0              585   747   692
994.40/994.56	c 1981950        9018411        180139833      930196951      898            0              579   737   688
1088.99/1089.19	c 2180211        9764883        197158069      1021268513     898            0              552   722   680
1199.08/1199.24	c 2398298        10735039       217109545      1130083512     898            0              581   739   692
1311.97/1312.16	c 2638193        11807730       237496651      1244339047     898            0              561   742   681
1450.16/1450.35	c 2902077        12915442       262271691      1376738809     898            0              558   731   685
1598.25/1598.46	c 3192349        14419782       289171359      1511862394     898            0              566   734   682
1767.63/1767.81	c 3511648        15976261       319839083      1670489620     898            0              602   736   683
1918.81/1919.08	c 3862876        17577034       347602447      1812538678     898            0              556   761   708
2107.09/2107.39	c 4249226        19262272       381500699      1990914969     898            0              623   722   674
2316.57/2316.86	c 4674211        21564032       419762627      2180009799     898            0              596   746   686

Verifier Data 

One s line expected

Watcher Data 

runsolver Copyright (C) 2010-2013 Olivier ROUSSEL

This is runsolver version 3.3.7 (svn: 2553)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4243121-1500670879/watcher-4243121-1500670879 -o /tmp/evaluation-result-4243121-1500670879/solver-4243121-1500670879 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node138-1500648173-9855 --watchdog 2560 HOME/cosoco -lc -model -restarts HOME/instance-4243121-1500670879.xml 

pid=12602
running on 8 cores: 1,3,5,7,9,11,13,15

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2500 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 8192 KiB


[startup+0 s]
/proc/loadavg: 2.11 2.06 2.05 3/243 12605
/proc/meminfo: memFree=32006620/32770624 swapFree=5908/201036
[pid=12605] ppid=12602 vsize=27912 CPUtime=0 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 1190 0 0 0 0 0 0 0 20 0 1 0 2354753 28581888 963 33554432000 4194304 5970956 140733429067712 140733429062872 140444978928613 0 0 4096 0 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 6978 1029 715 434 0 342 0

[startup+0.100151 s]
/proc/loadavg: 2.11 2.06 2.05 3/243 12605
/proc/meminfo: memFree=32006620/32770624 swapFree=5908/201036
[pid=12605] ppid=12602 vsize=32008 CPUtime=0.09 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 2282 0 0 0 9 0 0 0 20 0 1 0 2354753 32776192 2085 33554432000 4194304 5970956 140733429067712 140733429064408 4970936 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8002 2085 767 434 0 1366 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 32008

[startup+0.200312 s]
/proc/loadavg: 2.11 2.06 2.05 3/243 12605
/proc/meminfo: memFree=32006620/32770624 swapFree=5908/201036
[pid=12605] ppid=12602 vsize=32536 CPUtime=0.19 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 2417 0 0 0 19 0 0 0 20 0 1 0 2354753 33316864 2217 33554432000 4194304 5970956 140733429067712 140733429064408 4974412 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8134 2217 770 434 0 1498 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 32536

[startup+0.300239 s]
/proc/loadavg: 2.11 2.06 2.05 3/243 12605
/proc/meminfo: memFree=32006620/32770624 swapFree=5908/201036
[pid=12605] ppid=12602 vsize=32668 CPUtime=0.29 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 2439 0 0 0 29 0 0 0 20 0 1 0 2354753 33452032 2217 33554432000 4194304 5970956 140733429067712 140733429065048 4970963 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8167 2217 770 434 0 1531 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 32668

[startup+0.700227 s]
/proc/loadavg: 2.11 2.06 2.05 3/243 12605
/proc/meminfo: memFree=32006620/32770624 swapFree=5908/201036
[pid=12605] ppid=12602 vsize=32800 CPUtime=0.69 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 2469 0 0 0 69 0 0 0 20 0 1 0 2354753 33587200 2283 33554432000 4194304 5970956 140733429067712 140733429065048 4977871 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8200 2283 770 434 0 1564 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 32800

[startup+1.50021 s]
/proc/loadavg: 2.11 2.06 2.05 3/244 12606
/proc/meminfo: memFree=32000280/32770624 swapFree=5908/201036
[pid=12605] ppid=12602 vsize=32800 CPUtime=1.49 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 2479 0 0 0 149 0 0 0 20 0 1 0 2354753 33587200 2283 33554432000 4194304 5970956 140733429067712 140733429065048 4976581 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8200 2283 770 434 0 1564 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 32800

[startup+3.1002 s]
/proc/loadavg: 2.10 2.06 2.05 3/244 12606
/proc/meminfo: memFree=31999520/32770624 swapFree=5908/201036
[pid=12605] ppid=12602 vsize=32800 CPUtime=3.09 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 3939 0 0 0 309 0 0 0 20 0 1 0 2354753 33587200 2320 33554432000 4194304 5970956 140733429067712 140733429065048 4974816 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8200 2320 770 434 0 1564 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 32800

[startup+6.30021 s]
/proc/loadavg: 2.10 2.06 2.05 3/244 12606
/proc/meminfo: memFree=31999084/32770624 swapFree=5908/201036
[pid=12605] ppid=12602 vsize=32932 CPUtime=6.29 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 5411 0 0 0 629 0 0 0 20 0 1 0 2354753 33722368 2327 33554432000 4194304 5970956 140733429067712 140733429065048 5098896 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2327 770 434 0 1597 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 32932

[startup+12.7002 s]
/proc/loadavg: 2.10 2.06 2.05 3/244 12606
/proc/meminfo: memFree=31998712/32770624 swapFree=5908/201036
[pid=12605] ppid=12602 vsize=32932 CPUtime=12.69 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 6876 0 0 0 1269 0 0 0 20 0 1 0 2354753 33722368 2331 33554432000 4194304 5970956 140733429067712 140733429065048 4977712 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2331 770 434 0 1597 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 32932

[startup+25.5003 s]
/proc/loadavg: 2.15 2.07 2.05 3/244 12606
/proc/meminfo: memFree=31998560/32770624 swapFree=5908/201036
[pid=12605] ppid=12602 vsize=32932 CPUtime=25.48 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 8801 0 0 0 2548 0 0 0 20 0 1 0 2354753 33722368 2337 33554432000 4194304 5970956 140733429067712 140733429065048 4969894 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2337 770 434 0 1597 0
Current children cumulated CPU time (s) 25.48
Current children cumulated vsize (KiB) 32932

[startup+51.1003 s]
/proc/loadavg: 2.10 2.06 2.05 3/244 12606
/proc/meminfo: memFree=31998560/32770624 swapFree=5908/201036
[pid=12605] ppid=12602 vsize=32932 CPUtime=51.08 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 9269 0 0 0 5108 0 0 0 20 0 1 0 2354753 33722368 2340 33554432000 4194304 5970956 140733429067712 140733429065048 5322960 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2340 770 434 0 1597 0
Current children cumulated CPU time (s) 51.08
Current children cumulated vsize (KiB) 32932

[startup+102.3 s]
/proc/loadavg: 2.04 2.05 2.05 3/244 12607
/proc/meminfo: memFree=31997684/32770624 swapFree=5908/201036
[pid=12605] ppid=12602 vsize=32932 CPUtime=102.28 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 9732 0 0 0 10228 0 0 0 20 0 1 0 2354753 33722368 2342 33554432000 4194304 5970956 140733429067712 140733429065048 4978176 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2342 770 434 0 1597 0
Current children cumulated CPU time (s) 102.28
Current children cumulated vsize (KiB) 32932

[startup+162.3 s]
/proc/loadavg: 2.07 2.06 2.05 3/244 12608
/proc/meminfo: memFree=31998724/32770624 swapFree=5908/201288
[pid=12605] ppid=12602 vsize=32932 CPUtime=162.27 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 10195 0 0 0 16227 0 0 0 20 0 1 0 2354753 33722368 2343 33554432000 4194304 5970956 140733429067712 140733429065048 4970584 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2343 770 434 0 1597 0
Current children cumulated CPU time (s) 162.27
Current children cumulated vsize (KiB) 32932

[startup+222.308 s]
/proc/loadavg: 2.03 2.05 2.05 3/244 12609
/proc/meminfo: memFree=31998508/32770624 swapFree=5908/201288
[pid=12605] ppid=12602 vsize=32932 CPUtime=222.27 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 10661 0 0 0 22227 0 0 0 20 0 1 0 2354753 33722368 2343 33554432000 4194304 5970956 140733429067712 140733429065048 4970912 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2343 770 434 0 1597 0
Current children cumulated CPU time (s) 222.27
Current children cumulated vsize (KiB) 32932

[startup+282.3 s]
/proc/loadavg: 2.01 2.04 2.05 3/243 12610
/proc/meminfo: memFree=31998356/32770624 swapFree=5908/201288
[pid=12605] ppid=12602 vsize=32932 CPUtime=282.26 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 11121 0 0 0 28226 0 0 0 20 0 1 0 2354753 33722368 2344 33554432000 4194304 5970956 140733429067712 140733429065048 4977781 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2344 770 434 0 1597 0
Current children cumulated CPU time (s) 282.26
Current children cumulated vsize (KiB) 32932

[startup+342.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/243 12611
/proc/meminfo: memFree=31998204/32770624 swapFree=5908/201288
[pid=12605] ppid=12602 vsize=32932 CPUtime=342.25 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 11592 0 0 0 34225 0 0 0 20 0 1 0 2354753 33722368 2344 33554432000 4194304 5970956 140733429067712 140733429065048 5098896 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2344 770 434 0 1597 0
Current children cumulated CPU time (s) 342.25
Current children cumulated vsize (KiB) 32932

[startup+402.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/243 12612
/proc/meminfo: memFree=31998628/32770624 swapFree=5908/201288
[pid=12605] ppid=12602 vsize=32932 CPUtime=402.25 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 12058 0 0 0 40225 0 0 0 20 0 1 0 2354753 33722368 2344 33554432000 4194304 5970956 140733429067712 140733429065048 4977120 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2344 770 434 0 1597 0
Current children cumulated CPU time (s) 402.25
Current children cumulated vsize (KiB) 32932

[startup+462.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/242 12613
/proc/meminfo: memFree=31999640/32770624 swapFree=5908/201508
[pid=12605] ppid=12602 vsize=32932 CPUtime=462.24 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 12521 0 0 0 46224 0 0 0 20 0 1 0 2354753 33722368 2344 33554432000 4194304 5970956 140733429067712 140733429065048 4974143 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2344 770 434 0 1597 0
Current children cumulated CPU time (s) 462.24
Current children cumulated vsize (KiB) 32932

[startup+522.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/243 12615
/proc/meminfo: memFree=31998348/32770624 swapFree=5908/201508
[pid=12605] ppid=12602 vsize=32932 CPUtime=522.24 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 12988 0 0 0 52224 0 0 0 20 0 1 0 2354753 33722368 2344 33554432000 4194304 5970956 140733429067712 140733429065048 4977787 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2344 770 434 0 1597 0
Current children cumulated CPU time (s) 522.24
Current children cumulated vsize (KiB) 32932

[startup+582.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/243 12623
/proc/meminfo: memFree=31999036/32770624 swapFree=5908/201548
[pid=12605] ppid=12602 vsize=32932 CPUtime=582.23 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 13453 0 0 0 58223 0 0 0 20 0 1 0 2354753 33722368 2344 33554432000 4194304 5970956 140733429067712 140733429065048 4969902 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2344 770 434 0 1597 0
Current children cumulated CPU time (s) 582.23
Current children cumulated vsize (KiB) 32932


################
# More data... #
################


[startup+1542.31 s]
/proc/loadavg: 2.00 2.01 2.05 3/243 12713
/proc/meminfo: memFree=30643688/32770624 swapFree=5908/202076
[pid=12605] ppid=12602 vsize=32932 CPUtime=1542.16 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 19440 0 0 0 154215 1 0 0 20 0 1 0 2354753 33722368 2349 33554432000 4194304 5970956 140733429067712 140733429065048 4978463 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2349 770 434 0 1597 0
Current children cumulated CPU time (s) 1542.16
Current children cumulated vsize (KiB) 32932

[startup+1602.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/243 12714
/proc/meminfo: memFree=30643764/32770624 swapFree=5908/202076
[pid=12605] ppid=12602 vsize=32932 CPUtime=1602.14 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 19912 0 0 0 160213 1 0 0 20 0 1 0 2354753 33722368 2349 33554432000 4194304 5970956 140733429067712 140733429065048 4977120 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2349 770 434 0 1597 0
Current children cumulated CPU time (s) 1602.14
Current children cumulated vsize (KiB) 32932

[startup+1662.3 s]
/proc/loadavg: 2.05 2.03 2.05 3/243 12716
/proc/meminfo: memFree=30644632/32770624 swapFree=5908/202288
[pid=12605] ppid=12602 vsize=32932 CPUtime=1662.14 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 20148 0 0 0 166213 1 0 0 20 0 1 0 2354753 33722368 2349 33554432000 4194304 5970956 140733429067712 140733429065048 4977871 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2349 770 434 0 1597 0
Current children cumulated CPU time (s) 1662.14
Current children cumulated vsize (KiB) 32932

[startup+1722.3 s]
/proc/loadavg: 2.02 2.02 2.05 3/243 12717
/proc/meminfo: memFree=30644112/32770624 swapFree=5908/202304
[pid=12605] ppid=12602 vsize=32932 CPUtime=1722.13 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 20381 0 0 0 172212 1 0 0 20 0 1 0 2354753 33722368 2349 33554432000 4194304 5970956 140733429067712 140733429065048 4970950 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2349 770 434 0 1597 0
Current children cumulated CPU time (s) 1722.13
Current children cumulated vsize (KiB) 32932

[startup+1782.3 s]
/proc/loadavg: 2.01 2.02 2.05 3/243 12725
/proc/meminfo: memFree=30643460/32770624 swapFree=5908/202304
[pid=12605] ppid=12602 vsize=32932 CPUtime=1782.13 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 20850 0 0 0 178212 1 0 0 20 0 1 0 2354753 33722368 2349 33554432000 4194304 5970956 140733429067712 140733429065048 4977086 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2349 770 434 0 1597 0
Current children cumulated CPU time (s) 1782.13
Current children cumulated vsize (KiB) 32932

[startup+1842.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/243 12726
/proc/meminfo: memFree=30642676/32770624 swapFree=5908/202304
[pid=12605] ppid=12602 vsize=32932 CPUtime=1842.12 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 21056 0 0 0 184211 1 0 0 20 0 1 0 2354753 33722368 2349 33554432000 4194304 5970956 140733429067712 140733429065048 4978135 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2349 770 434 0 1597 0
Current children cumulated CPU time (s) 1842.12
Current children cumulated vsize (KiB) 32932

[startup+1902.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/243 12727
/proc/meminfo: memFree=30643004/32770624 swapFree=5908/202304
[pid=12605] ppid=12602 vsize=32932 CPUtime=1902.11 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 21260 0 0 0 190210 1 0 0 20 0 1 0 2354753 33722368 2349 33554432000 4194304 5970956 140733429067712 140733429065048 4970661 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2349 770 434 0 1597 0
Current children cumulated CPU time (s) 1902.11
Current children cumulated vsize (KiB) 32932

[startup+1962.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/243 12728
/proc/meminfo: memFree=30643444/32770624 swapFree=5908/202492
[pid=12605] ppid=12602 vsize=32932 CPUtime=1962.11 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 21728 0 0 0 196210 1 0 0 20 0 1 0 2354753 33722368 2349 33554432000 4194304 5970956 140733429067712 140733429065048 4970015 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2349 770 434 0 1597 0
Current children cumulated CPU time (s) 1962.11
Current children cumulated vsize (KiB) 32932

[startup+2022.3 s]
/proc/loadavg: 2.05 2.03 2.05 3/243 12729
/proc/meminfo: memFree=30642480/32770624 swapFree=5908/202492
[pid=12605] ppid=12602 vsize=32932 CPUtime=2022.1 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 21953 0 0 0 202209 1 0 0 20 0 1 0 2354753 33722368 2349 33554432000 4194304 5970956 140733429067712 140733429065048 4976739 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2349 770 434 0 1597 0
Current children cumulated CPU time (s) 2022.1
Current children cumulated vsize (KiB) 32932

[startup+2082.3 s]
/proc/loadavg: 2.07 2.04 2.05 3/243 12730
/proc/meminfo: memFree=30642940/32770624 swapFree=5908/202492
[pid=12605] ppid=12602 vsize=32932 CPUtime=2082.1 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 22180 0 0 0 208209 1 0 0 20 0 1 0 2354753 33722368 2349 33554432000 4194304 5970956 140733429067712 140733429065048 4978135 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2349 770 434 0 1597 0
Current children cumulated CPU time (s) 2082.1
Current children cumulated vsize (KiB) 32932

[startup+2142.3 s]
/proc/loadavg: 2.03 2.03 2.05 3/243 12731
/proc/meminfo: memFree=30642704/32770624 swapFree=5908/202492
[pid=12605] ppid=12602 vsize=32932 CPUtime=2142.09 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 22648 0 0 0 214208 1 0 0 20 0 1 0 2354753 33722368 2349 33554432000 4194304 5970956 140733429067712 140733429065048 4976774 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2349 770 434 0 1597 0
Current children cumulated CPU time (s) 2142.09
Current children cumulated vsize (KiB) 32932

[startup+2202.31 s]
/proc/loadavg: 2.01 2.03 2.05 3/243 12732
/proc/meminfo: memFree=30642016/32770624 swapFree=5908/202492
[pid=12605] ppid=12602 vsize=32932 CPUtime=2202.09 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 22878 0 0 0 220208 1 0 0 20 0 1 0 2354753 33722368 2349 33554432000 4194304 5970956 140733429067712 140733429065048 5322656 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2349 770 434 0 1597 0
Current children cumulated CPU time (s) 2202.09
Current children cumulated vsize (KiB) 32932

[startup+2262.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/243 12733
/proc/meminfo: memFree=30643504/32770624 swapFree=5908/202676
[pid=12605] ppid=12602 vsize=32932 CPUtime=2262.08 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 23115 0 0 0 226207 1 0 0 20 0 1 0 2354753 33722368 2349 33554432000 4194304 5970956 140733429067712 140733429065048 4974167 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2349 770 434 0 1597 0
Current children cumulated CPU time (s) 2262.08
Current children cumulated vsize (KiB) 32932

[startup+2322.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/243 12734
/proc/meminfo: memFree=30642628/32770624 swapFree=5908/202676
[pid=12605] ppid=12602 vsize=32932 CPUtime=2322.07 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 23583 0 0 0 232206 1 0 0 20 0 1 0 2354753 33722368 2349 33554432000 4194304 5970956 140733429067712 140733429065048 4978607 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2349 770 434 0 1597 0
Current children cumulated CPU time (s) 2322.07
Current children cumulated vsize (KiB) 32932

[startup+2382.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/243 12742
/proc/meminfo: memFree=30643376/32770624 swapFree=5908/202676
[pid=12605] ppid=12602 vsize=32932 CPUtime=2382.06 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 23801 0 0 0 238205 1 0 0 20 0 1 0 2354753 33722368 2349 33554432000 4194304 5970956 140733429067712 140733429065048 4976570 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2349 770 434 0 1597 0
Current children cumulated CPU time (s) 2382.06
Current children cumulated vsize (KiB) 32932



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+2400.31 s]
/proc/loadavg: 2.00 2.01 2.05 3/243 12742
/proc/meminfo: memFree=30643376/32770624 swapFree=5908/202676
[pid=12605] ppid=12602 vsize=32932 CPUtime=2400.07 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 24007 0 0 0 240006 1 0 0 20 0 1 0 2354753 33722368 2349 33554432000 4194304 5970956 140733429067712 140733429065048 4978199 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2349 770 434 0 1597 0
Current children cumulated CPU time (s) 2400.07
Current children cumulated vsize (KiB) 32932

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended.
??? exit watcher main loop
??? sigkill at end of watcher main loop
??? kill 12605 sig 9
??? end of watcher thread
Dumping a history of the last processes samples

[startup+2400.31 s]
/proc/loadavg: 2.00 2.01 2.05 3/243 12742
/proc/meminfo: memFree=30643376/32770624 swapFree=5908/202676
[pid=12605] ppid=12602 vsize=32932 CPUtime=2400.07 cores=1,3,5,7,9,11,13,15
/proc/12605/stat : 12605 (cosoco) R 12602 12605 9822 0 -1 4202496 24007 0 0 0 240006 1 0 0 20 0 1 0 2354753 33722368 2349 33554432000 4194304 5970956 140733429067712 140733429065048 4978199 0 0 4096 8405026 0 0 0 17 13 0 0 0 0 0 8070896 8075904 33316864 140733429068993 140733429069123 140733429069123 140733429071822 0
/proc/12605/statm: 8233 2349 770 434 0 1597 0
Current children cumulated CPU time (s) 2400.07
Current children cumulated vsize (KiB) 32932

Child status: 1
Real time (s): 2400.31
CPU time (s): 2400.08
CPU user time (s): 2400.06
CPU system time (s): 0.011998
CPU usage (%): 99.9903
Max. virtual memory (cumulated for all children) (KiB): 32932

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2400.06
system time used= 0.011998
maximum resident set size= 9440
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 24036
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 14
involuntary context switches= 120624

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 7.39588 second user time and 10.1568 second system time

The end

Launcher Data 

Begin job on node138 at 2017-07-21 23:01:19
IDJOB=4243121
IDBENCH=138962
IDSOLVER=2620
FILE ID=node138/4243121-1500670879
RUNJOBID= node138-1500648173-9855
PBS_JOBID= 20623282
Free space on /tmp= 61992 MiB

SOLVER NAME= cosoco 1.1
BENCH NAME= XCSP17/LatinSquare/LatinSquare-m1-gp/qwh-o30-h375-17.xml
COMMAND LINE= DIR/cosoco -lc -model -restarts BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4243121-1500670879/watcher-4243121-1500670879 -o /tmp/evaluation-result-4243121-1500670879/solver-4243121-1500670879 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node138-1500648173-9855 --watchdog 2560  HOME/cosoco -lc -model -restarts HOME/instance-4243121-1500670879.xml

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 8

MD5SUM BENCH= 946d300933e85b0b5e5d11b6e9e445b0
RANDOM SEED=359441238

node138.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1867.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 4
apicid		: 1
initial apicid	: 1
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 4
apicid		: 17
initial apicid	: 17
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 4
apicid		: 3
initial apicid	: 3
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 4
apicid		: 19
initial apicid	: 19
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 4
apicid		: 5
initial apicid	: 5
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 4
apicid		: 21
initial apicid	: 21
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 4
apicid		: 7
initial apicid	: 7
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 4
apicid		: 23
initial apicid	: 23
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        32006376 kB
MemAvailable:   32008296 kB
Buffers:           38680 kB
Cached:           273108 kB
SwapCached:            0 kB
Active:           176532 kB
Inactive:         209628 kB
Active(anon):      77016 kB
Inactive(anon):     8592 kB
Active(file):      99516 kB
Inactive(file):   201036 kB
Unevictable:        5908 kB
Mlocked:            5908 kB
SwapTotal:      67108860 kB
SwapFree:       67108860 kB
Dirty:              2440 kB
Writeback:             0 kB
AnonPages:        165656 kB
Mapped:            32596 kB
Shmem:              8852 kB
Slab:              66680 kB
SReclaimable:      34404 kB
SUnreclaim:        32276 kB
KernelStack:        4272 kB
PageTables:         5788 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     343528 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:    108544 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      140672 kB
DirectMap2M:    33404928 kB

Free space on /tmp at the end= 61988 MiB
End job on node138 at 2017-07-21 23:41:19