Trace number 4242020

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
R1: cosoco 1.1? (TO) 2400.02 2400.21

General information on the benchmark

NameOrtholatin/Ortholatin-m1-s1/
Ortholatin-024.xml
MD5SUM23eb83e679dec689804004fda8e6ed87
Bench CategoryCSP (decision problem)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1728
Number of constraints581
Number of domains2
Minimum domain size24
Maximum domain size576
Distribution of domain sizes[{"size":24,"count":1152},{"size":576,"count":576}]
Minimum variable degree2
Maximum variable degree3
Distribution of variable degrees[{"degree":2,"count":1680},{"degree":3,"count":48}]
Minimum constraint arity3
Maximum constraint arity576
Distribution of constraint arities[{"arity":3,"count":576},{"arity":24,"count":2},{"arity":576,"count":3}]
Number of extensional constraints576
Number of intensional constraints0
Distribution of constraint types[{"type":"extension","count":576},{"type":"allDifferent","count":3},{"type":"instantiation","count":2}]
Optimization problemNO
Type of objective

Solver Data 

0.00/0.08	c root propagations     : 17756       
0.00/0.08	c restarts enabled, print stats at every restarts
0.00/0.08	c
0.00/0.08	c
0.00/0.08	c
0.00/0.08	c This is cosoco 1.0 --  
0.00/0.08	c
0.00/0.08	c ========================================[ Problem Statistics ]===========================================
0.00/0.08	c |                                                                                                       
0.00/0.08	c |  Parse time        :         0.03 s 
0.00/0.08	c |
0.00/0.08	c |               Variables: 1728
0.00/0.08	c |            Domain Sizes: 24..576
0.00/0.08	c |
0.00/0.08	c |             Constraints: 721
0.00/0.08	c |                   Arity: 1..576  -- Unary: 48  -- Ternary: 576
0.00/0.08	c | 
0.00/0.08	c |                   Types: 
0.00/0.08	c |                          All Different: 97
0.00/0.08	c |                          Extension: 576  (nb tuples: 576..576) -- (shared: 575)
0.00/0.08	c |                          Unary: 48
0.00/0.08	c |                          
0.00/0.08	c =========================================================================================================
0.00/0.08	c conflicts      decisions      filterCalls    propagations   Root Prop.     Nogoods        min/max/avg depth
0.08/0.13	c 101            355            5704           59460          716            0              201   255   225
0.18/0.25	c 211            976            16196          188200         716            0              341   358   346
0.28/0.32	c 332            1292           20963          214716         716            0              196   196   194
0.28/0.39	c 465            1623           25792          241855         716            0              199   199   197
0.38/0.47	c 611            1970           30659          269572         716            0              202   202   200
0.48/0.54	c 771            2334           35710          297935         716            0              205   205   203
0.58/0.62	c 947            2717           40793          326881         716            0              208   208   206
0.68/0.70	c 1140           3156           46306          361020         716            0              238   247   240
0.68/0.78	c 1352           3620           52025          397202         716            0              250   253   250
0.78/0.87	c 1585           4115           57955          436168         716            0              250   259   255
0.88/0.96	c 1841           4635           63536          474242         716            0              262   265   261
0.98/1.04	c 2122           5186           68814          513526         716            0              268   271   267
1.09/1.19	c 2431           5805           78437          569223         716            0              266   277   273
1.28/1.30	c 2770           6426           85035          612784         716            0              259   277   259
1.38/1.47	c 3142           7128           94460          665597         716            0              307   331   322
1.69/1.71	c 3551           7971           108241         753887         716            0              316   409   390
1.99/2.00	c 4000           9086           128255         958650         716            0              331   399   346
2.09/2.14	c 4493           9813           135637         994319         716            0              232   235   232
2.20/2.27	c 5035           10601          142736         1029697        716            0              244   247   244
2.39/2.49	c 5631           11533          152866         1086881        716            0              259   337   283
2.69/2.79	c 6286           12693          169456         1223941        716            0              1   399   379  
2.89/2.99	c 7006           13758          180622         1281498        716            0              343   346   343
3.30/3.40	c 7798           15172          202969         1456324        716            0              349   433   416
3.59/3.66	c 8669           16456          217127         1533842        716            0              319   409   403
4.40/4.45	c 9627           18422          265906         1849499        716            0              359   407   398
4.69/4.72	c 10680          19785          278737         1912774        716            0              302   311   305
5.09/5.16	c 11838          21492          299333         2045089        716            0              304   468   429
5.59/5.69	c 13111          23313          324845         2188879        716            0              369   462   423
6.49/6.50	c 14511          26339          378291         2724354        716            0              408   491   473
6.89/6.93	c 16051          28327          398182         2829605        716            0              366   393   375
7.49/7.51	c 17745          30603          424322         2988962        716            0              347   477   429
8.09/8.10	c 19608          33436          453796         3311162        716            0              315   475   432
8.89/8.92	c 21657          36356          494960         3608295        716            0              370   516   474
10.18/10.26	c 23910          39788          566237         4095712        716            0              356   465   458
10.88/10.93	c 26388          43481          606139         4539910        716            0              343   391   383
12.28/12.38	c 29113          47703          685180         5053522        716            0              256   560   435
13.88/13.92	c 32110          53830          792317         6223718        716            0              311   426   376
15.08/15.16	c 35406          58117          856652         6567922        716            0              320   379   366
16.08/16.17	c 39031          63687          916925         7302688        716            0              268   379   369
17.38/17.47	c 43018          69097          974357         7815271        716            0              270   365   349
18.88/18.92	c 47403          74848          1042134        8406135        716            0              288   351   337
20.38/20.44	c 52226          84237          1130690        9798287        716            0              1   395   358  
21.78/21.88	c 57531          90969          1184920        10197201       716            0              234   556   433
24.78/24.81	c 63366          99619          1363026        11300472       716            0              218   457   266
26.98/27.00	c 69784          107961         1477103        12135041       716            0              253   367   358
30.48/30.57	c 76843          118794         1692608        13971450       716            0              276   563   435
32.28/32.37	c 84607          130217         1772283        15214136       716            0              226   425   408
34.28/34.33	c 93147          141208         1859120        15958302       716            0              337   424   409
36.78/36.88	c 102541         155206         2000468        17815957       716            0              271   425   408
40.99/41.08	c 112874         171651         2271835        20377137       716            0              204   479   410
46.79/46.89	c 124240         189628         2624499        22956093       716            0              249   427   411
51.09/51.18	c 136742         205439         2801678        24140610       716            0              276   941   629
54.29/54.36	c 150494         222254         2925206        25246444       716            0              231   426   410
58.78/58.82	c 165621         241668         3144423        26777396       716            0              224   425   380
64.18/64.28	c 182260         264042         3453412        28653863       716            0              260   394   380
69.58/69.65	c 200562         287440         3754175        30392790       716            0              201   362   351
74.78/74.88	c 220694         312432         4038821        32227466       716            0              185   594   392
81.38/81.50	c 242839         343138         4379494        35384164       716            0              197   878   494
87.57/87.60	c 267198         371841         4669626        36862749       716            0              201   675   401
92.07/92.17	c 293992         403276         4867508        38518901       716            0              227   423   414
101.97/102.05	c 323465         442543         5457041        42601084       716            0              227   335   324
113.47/113.55	c 355885         490574         5984638        50628672       716            0              233   1031   869
123.18/123.30	c 391547         535690         6382220        56467844       716            0              242   911   744
134.28/134.37	c 430775         586863         6906534        61702238       716            0              282   774   675
145.18/145.25	c 473925         655786         7522023        71224719       716            0              243   737   648
155.47/155.60	c 521390         717700         7994942        76801095       716            0              202   1080   665
165.57/165.63	c 573601         786298         8450912        81890637       716            0              214   504   468
181.27/181.38	c 631033         858024         9286284        87482574       716            0              311   408   397
195.27/195.38	c 694208         944177         9870584        95361868       716            0              194   691   642
211.66/211.78	c 763700         1040256        10718688       105896179      716            0              216   831   665
227.67/227.79	c 840141         1142810        11485817       113266985      716            0              247   512   488
256.96/257.03	c 924226         1262858        13196930       127242896      716            0              301   502   468
282.06/282.15	c 1016719        1381748        14593903       136637925      716            0              315   460   448
313.25/313.36	c 1118461        1527331        16518386       151120947      716            0              187   531   415
336.25/336.35	c 1230377        1671217        17628080       162747733      716            0              221   701   567
367.25/367.33	c 1353484        1824801        19265297       174157889      716            0              231   573   550
398.26/398.38	c 1488901        1989574        20729994       187016296      716            0              252   617   595
429.25/429.37	c 1637859        2184900        22193308       202560821      716            0              248   681   576
490.24/490.34	c 1801712        2440210        26017816       238101618      716            0              326   572   545
537.74/537.80	c 1981950        2696271        28454959       261131737      716            0              244   446   434
605.23/605.37	c 2180211        2981844        32460026       291811992      716            0              209   666   559
647.33/647.44	c 2398298        3256805        34379186       309978988      716            0              252   491   477
698.33/698.44	c 2638193        3585671        37165177       337884914      716            0              251   533   520
751.82/751.90	c 2902077        3938871        39916662       363620190      716            0              238   384   366
802.52/802.69	c 3192349        4278002        42241962       382487625      716            0              329   344   334
888.62/888.76	c 3511648        4712023        46730326       423117152      716            0              278   569   544
992.12/992.26	c 3862876        5230770        52220433       475302020      716            0              231   855   564
1074.90/1075.03	c 4249226        5764995        56790371       533915449      716            0              332   550   536
1165.20/1165.34	c 4674211        6350779        61089055       586872218      716            0              1   663   603  
1261.19/1261.31	c 5141694        6964238        64829685       636664864      716            0              413   950   650
1366.98/1367.14	c 5655925        7688276        70403048       696921821      716            0              224   524   513
1488.58/1488.77	c 6221579        8449143        76967437       756723296      716            0              296   578   564
1634.76/1634.92	c 6843798        9292706        84102424       845776375      716            0              274   1012   711
1835.76/1835.93	c 7528238        10266821       94333858       923839078      716            0              221   530   518
1974.45/1974.64	c 8281122        11192623       100813240      979734692      716            0              365   476   464
2288.20/2288.41	c 9109294        12675388       117448636      1268742705     716            0              247   800   772

Verifier Data 

One s line expected

Watcher Data 

runsolver Copyright (C) 2010-2013 Olivier ROUSSEL

This is runsolver version 3.3.7 (svn: 2553)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4242020-1500656640/watcher-4242020-1500656640 -o /tmp/evaluation-result-4242020-1500656640/solver-4242020-1500656640 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node124-1500648174-2481 --watchdog 2560 HOME/cosoco -lc -model -restarts HOME/instance-4242020-1500656640.xml 

pid=3644
running on 8 cores: 0,2,4,6,8,10,12,14

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2500 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 8192 KiB


[startup+0 s]
/proc/loadavg: 2.00 2.01 2.05 3/242 3647
/proc/meminfo: memFree=32042624/32770624 swapFree=5796/212092
[pid=3647] ppid=3644 vsize=27520 CPUtime=0 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 1069 0 0 0 0 0 0 0 20 0 1 0 931752 28180480 898 33554432000 4194304 5970956 140724310328656 140724310325304 140568460962789 0 0 4096 0 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 6880 898 702 434 0 244 0

[startup+0.100121 s]
/proc/loadavg: 2.00 2.01 2.05 3/242 3647
/proc/meminfo: memFree=32042624/32770624 swapFree=5796/212092
[pid=3647] ppid=3644 vsize=42140 CPUtime=0.08 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 4802 0 0 0 8 0 0 0 20 0 1 0 931752 43151360 4594 33554432000 4194304 5970956 140724310328656 140724310326264 140568460962789 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 10535 4594 734 434 0 3899 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 42140

[startup+0.200281 s]
/proc/loadavg: 2.00 2.01 2.05 3/242 3647
/proc/meminfo: memFree=32042624/32770624 swapFree=5796/212092
[pid=3647] ppid=3644 vsize=44136 CPUtime=0.18 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 5309 0 0 0 18 0 0 0 20 0 1 0 931752 45195264 5122 33554432000 4194304 5970956 140724310328656 140724310325576 140568460962789 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 11034 5122 776 434 0 4398 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 44136

[startup+0.300281 s]
/proc/loadavg: 2.00 2.01 2.05 3/242 3647
/proc/meminfo: memFree=32042624/32770624 swapFree=5796/212092
[pid=3647] ppid=3644 vsize=44796 CPUtime=0.28 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 5455 0 0 0 28 0 0 0 20 0 1 0 931752 45871104 5254 33554432000 4194304 5970956 140724310328656 140724310325992 5093256 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 11199 5254 776 434 0 4563 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 44796

[startup+0.700226 s]
/proc/loadavg: 2.00 2.01 2.05 3/242 3647
/proc/meminfo: memFree=32042624/32770624 swapFree=5796/212092
[pid=3647] ppid=3644 vsize=44796 CPUtime=0.68 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 5455 0 0 0 68 0 0 0 20 0 1 0 931752 45871104 5254 33554432000 4194304 5970956 140724310328656 140724310325992 5098928 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 11199 5254 776 434 0 4563 0
Current children cumulated CPU time (s) 0.68
Current children cumulated vsize (KiB) 44796

[startup+1.50022 s]
/proc/loadavg: 2.00 2.01 2.05 3/243 3648
/proc/meminfo: memFree=32025096/32770624 swapFree=5796/212076
[pid=3647] ppid=3644 vsize=45060 CPUtime=1.48 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 5548 0 0 0 148 0 0 0 20 0 1 0 931752 46141440 5386 33554432000 4194304 5970956 140724310328656 140724310325992 4976560 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 11265 5386 776 434 0 4629 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 45060

[startup+3.10019 s]
/proc/loadavg: 2.00 2.01 2.05 3/243 3648
/proc/meminfo: memFree=32023228/32770624 swapFree=5796/212076
[pid=3647] ppid=3644 vsize=46512 CPUtime=3.09 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 10176 0 0 0 308 1 0 0 20 0 1 0 931752 47628288 5743 33554432000 4194304 5970956 140724310328656 140724310325992 4934506 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 11628 5743 776 434 0 4992 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 46512

[startup+6.30017 s]
/proc/loadavg: 1.92 2.00 2.04 3/243 3648
/proc/meminfo: memFree=32022132/32770624 swapFree=5796/212076
[pid=3647] ppid=3644 vsize=47040 CPUtime=6.29 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 14704 0 0 0 628 1 0 0 20 0 1 0 931752 48168960 5852 33554432000 4194304 5970956 140724310328656 140724310325464 4974816 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 11760 5852 776 434 0 5124 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 47040

[startup+12.7002 s]
/proc/loadavg: 1.93 2.00 2.04 3/243 3648
/proc/meminfo: memFree=32021388/32770624 swapFree=5796/212076
[pid=3647] ppid=3644 vsize=48344 CPUtime=12.68 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 19455 0 0 0 1267 1 0 0 20 0 1 0 931752 49504256 6154 33554432000 4194304 5970956 140724310328656 140724310325992 4930648 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 12086 6154 777 434 0 5450 0
Current children cumulated CPU time (s) 12.68
Current children cumulated vsize (KiB) 48344

[startup+25.5003 s]
/proc/loadavg: 1.95 2.00 2.04 3/243 3648
/proc/meminfo: memFree=32020700/32770624 swapFree=5796/212124
[pid=3647] ppid=3644 vsize=48476 CPUtime=25.48 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 28892 0 0 0 2547 1 0 0 20 0 1 0 931752 49639424 6239 33554432000 4194304 5970956 140724310328656 140724310325992 4934556 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 12119 6239 777 434 0 5483 0
Current children cumulated CPU time (s) 25.48
Current children cumulated vsize (KiB) 48476

[startup+51.1003 s]
/proc/loadavg: 2.03 2.01 2.05 3/243 3648
/proc/meminfo: memFree=32019504/32770624 swapFree=5796/212176
[pid=3647] ppid=3644 vsize=50880 CPUtime=51.09 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 34386 0 0 0 5107 2 0 0 20 0 1 0 931752 52101120 6775 33554432000 4194304 5970956 140724310328656 140724310325432 4977138 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 12720 6775 777 434 0 6084 0
Current children cumulated CPU time (s) 51.09
Current children cumulated vsize (KiB) 50880

[startup+102.3 s]
/proc/loadavg: 2.07 2.03 2.05 3/243 3649
/proc/meminfo: memFree=32018064/32770624 swapFree=5796/212272
[pid=3647] ppid=3644 vsize=51408 CPUtime=102.28 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 35771 0 0 0 10226 2 0 0 20 0 1 0 931752 52641792 6897 33554432000 4194304 5970956 140724310328656 140724310325992 4934354 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 12852 6897 777 434 0 6216 0
Current children cumulated CPU time (s) 102.28
Current children cumulated vsize (KiB) 51408

[startup+162.3 s]
/proc/loadavg: 2.02 2.02 2.05 3/243 3650
/proc/meminfo: memFree=32017272/32770624 swapFree=5796/212336
[pid=3647] ppid=3644 vsize=52600 CPUtime=162.27 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 36716 0 0 0 16225 2 0 0 20 0 1 0 931752 53862400 7221 33554432000 4194304 5970956 140724310328656 140724310325992 4974167 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7221 777 434 0 6514 0
Current children cumulated CPU time (s) 162.27
Current children cumulated vsize (KiB) 52600

[startup+222.308 s]
/proc/loadavg: 2.01 2.02 2.05 3/243 3651
/proc/meminfo: memFree=32014880/32770624 swapFree=5796/212388
[pid=3647] ppid=3644 vsize=52600 CPUtime=222.27 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 37432 0 0 0 22225 2 0 0 20 0 1 0 931752 53862400 7243 33554432000 4194304 5970956 140724310328656 140724310325992 4978194 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7243 777 434 0 6514 0
Current children cumulated CPU time (s) 222.27
Current children cumulated vsize (KiB) 52600

[startup+282.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/243 3652
/proc/meminfo: memFree=32015676/32770624 swapFree=5796/212444
[pid=3647] ppid=3644 vsize=52600 CPUtime=282.26 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 37648 0 0 0 28224 2 0 0 20 0 1 0 931752 53862400 7243 33554432000 4194304 5970956 140724310328656 140724310325992 4934545 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7243 777 434 0 6514 0
Current children cumulated CPU time (s) 282.26
Current children cumulated vsize (KiB) 52600

[startup+342.3 s]
/proc/loadavg: 2.08 2.03 2.05 3/243 3653
/proc/meminfo: memFree=32015416/32770624 swapFree=5796/212488
[pid=3647] ppid=3644 vsize=52600 CPUtime=342.25 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 37877 0 0 0 34223 2 0 0 20 0 1 0 931752 53862400 7243 33554432000 4194304 5970956 140724310328656 140724310325992 5311662 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7243 777 434 0 6514 0
Current children cumulated CPU time (s) 342.25
Current children cumulated vsize (KiB) 52600

[startup+402.3 s]
/proc/loadavg: 2.03 2.02 2.05 3/243 3661
/proc/meminfo: memFree=32014840/32770624 swapFree=5796/212552
[pid=3647] ppid=3644 vsize=52600 CPUtime=402.25 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 38104 0 0 0 40223 2 0 0 20 0 1 0 931752 53862400 7243 33554432000 4194304 5970956 140724310328656 140724310325992 4976663 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7243 777 434 0 6514 0
Current children cumulated CPU time (s) 402.25
Current children cumulated vsize (KiB) 52600

[startup+462.3 s]
/proc/loadavg: 2.01 2.02 2.05 3/243 3662
/proc/meminfo: memFree=32014912/32770624 swapFree=5796/212592
[pid=3647] ppid=3644 vsize=52600 CPUtime=462.24 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 38330 0 0 0 46222 2 0 0 20 0 1 0 931752 53862400 7243 33554432000 4194304 5970956 140724310328656 140724310325992 4934354 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7243 777 434 0 6514 0
Current children cumulated CPU time (s) 462.24
Current children cumulated vsize (KiB) 52600

[startup+522.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/243 3663
/proc/meminfo: memFree=32015244/32770624 swapFree=5796/212636
[pid=3647] ppid=3644 vsize=52600 CPUtime=522.24 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 38551 0 0 0 52222 2 0 0 20 0 1 0 931752 53862400 7243 33554432000 4194304 5970956 140724310328656 140724310325992 4934186 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7243 777 434 0 6514 0
Current children cumulated CPU time (s) 522.24
Current children cumulated vsize (KiB) 52600

[startup+582.3 s]
/proc/loadavg: 2.06 2.03 2.05 3/243 3664
/proc/meminfo: memFree=32014492/32770624 swapFree=5796/212668
[pid=3647] ppid=3644 vsize=52600 CPUtime=582.24 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 38773 0 0 0 58222 2 0 0 20 0 1 0 931752 53862400 7243 33554432000 4194304 5970956 140724310328656 140724310325992 4977752 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7243 777 434 0 6514 0
Current children cumulated CPU time (s) 582.24
Current children cumulated vsize (KiB) 52600


################
# More data... #
################

[startup+1482.3 s]
/proc/loadavg: 2.01 2.02 2.05 3/243 3689
/proc/meminfo: memFree=32014448/32770624 swapFree=5796/213320
[pid=3647] ppid=3644 vsize=52600 CPUtime=1482.18 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 41727 0 0 0 148216 2 0 0 20 0 1 0 931752 53862400 7243 33554432000 4194304 5970956 140724310328656 140724310325992 4974327 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7243 777 434 0 6514 0
Current children cumulated CPU time (s) 1482.18
Current children cumulated vsize (KiB) 52600

[startup+1542.31 s]
/proc/loadavg: 2.00 2.01 2.05 3/243 3690
/proc/meminfo: memFree=32015020/32770624 swapFree=5796/213372
[pid=3647] ppid=3644 vsize=52600 CPUtime=1542.18 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 42007 0 0 0 154216 2 0 0 20 0 1 0 931752 53862400 7243 33554432000 4194304 5970956 140724310328656 140724310325992 5098899 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7243 777 434 0 6514 0
Current children cumulated CPU time (s) 1542.18
Current children cumulated vsize (KiB) 52600

[startup+1602.3 s]
/proc/loadavg: 2.10 2.04 2.05 3/244 3699
/proc/meminfo: memFree=32014632/32770624 swapFree=5796/213412
[pid=3647] ppid=3644 vsize=52600 CPUtime=1602.17 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 42196 0 0 0 160215 2 0 0 20 0 1 0 931752 53862400 7243 33554432000 4194304 5970956 140724310328656 140724310325992 4930544 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7243 777 434 0 6514 0
Current children cumulated CPU time (s) 1602.17
Current children cumulated vsize (KiB) 52600

[startup+1662.3 s]
/proc/loadavg: 2.04 2.04 2.05 3/244 3700
/proc/meminfo: memFree=32015400/32770624 swapFree=5796/213444
[pid=3647] ppid=3644 vsize=52600 CPUtime=1662.16 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 42435 0 0 0 166214 2 0 0 20 0 1 0 931752 53862400 7249 33554432000 4194304 5970956 140724310328656 140724310325992 4977752 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7249 777 434 0 6514 0
Current children cumulated CPU time (s) 1662.16
Current children cumulated vsize (KiB) 52600

[startup+1722.3 s]
/proc/loadavg: 2.01 2.03 2.05 3/244 3701
/proc/meminfo: memFree=32014796/32770624 swapFree=5796/213472
[pid=3647] ppid=3644 vsize=52600 CPUtime=1722.16 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 42582 0 0 0 172214 2 0 0 20 0 1 0 931752 53862400 7249 33554432000 4194304 5970956 140724310328656 140724310325992 4934366 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7249 777 434 0 6514 0
Current children cumulated CPU time (s) 1722.16
Current children cumulated vsize (KiB) 52600

[startup+1782.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/244 3702
/proc/meminfo: memFree=32013392/32770624 swapFree=5796/213508
[pid=3647] ppid=3644 vsize=52600 CPUtime=1782.15 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 42728 0 0 0 178213 2 0 0 20 0 1 0 931752 53862400 7249 33554432000 4194304 5970956 140724310328656 140724310325992 4978194 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7249 777 434 0 6514 0
Current children cumulated CPU time (s) 1782.15
Current children cumulated vsize (KiB) 52600

[startup+1842.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/244 3703
/proc/meminfo: memFree=32014988/32770624 swapFree=5796/213548
[pid=3647] ppid=3644 vsize=52600 CPUtime=1842.15 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 42949 0 0 0 184213 2 0 0 20 0 1 0 931752 53862400 7249 33554432000 4194304 5970956 140724310328656 140724310325992 4930611 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7249 777 434 0 6514 0
Current children cumulated CPU time (s) 1842.15
Current children cumulated vsize (KiB) 52600

[startup+1902.3 s]
/proc/loadavg: 2.03 2.03 2.05 3/243 3704
/proc/meminfo: memFree=32014272/32770624 swapFree=5796/213596
[pid=3647] ppid=3644 vsize=52600 CPUtime=1902.14 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 43098 0 0 0 190212 2 0 0 20 0 1 0 931752 53862400 7249 33554432000 4194304 5970956 140724310328656 140724310325992 4974299 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7249 777 434 0 6514 0
Current children cumulated CPU time (s) 1902.14
Current children cumulated vsize (KiB) 52600

[startup+1962.3 s]
/proc/loadavg: 2.01 2.02 2.05 3/243 3771
/proc/meminfo: memFree=32018668/32770624 swapFree=5796/213652
[pid=3647] ppid=3644 vsize=52600 CPUtime=1962.14 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 43246 0 0 0 196212 2 0 0 20 0 1 0 931752 53862400 7249 33554432000 4194304 5970956 140724310328656 140724310325992 4934354 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7249 777 434 0 6514 0
Current children cumulated CPU time (s) 1962.14
Current children cumulated vsize (KiB) 52600

[startup+2022.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/243 3903
/proc/meminfo: memFree=32020364/32770624 swapFree=5796/213780
[pid=3647] ppid=3644 vsize=52600 CPUtime=2022.13 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 43516 0 0 0 202211 2 0 0 20 0 1 0 931752 53862400 7256 33554432000 4194304 5970956 140724310328656 140724310325992 5327383 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7256 777 434 0 6514 0
Current children cumulated CPU time (s) 2022.13
Current children cumulated vsize (KiB) 52600

[startup+2082.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/243 3969
/proc/meminfo: memFree=32019636/32770624 swapFree=5796/213872
[pid=3647] ppid=3644 vsize=52600 CPUtime=2082.13 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 43677 0 0 0 208211 2 0 0 20 0 1 0 931752 53862400 7260 33554432000 4194304 5970956 140724310328656 140724310325992 4974816 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7260 777 434 0 6514 0
Current children cumulated CPU time (s) 2082.13
Current children cumulated vsize (KiB) 52600

[startup+2142.3 s]
/proc/loadavg: 2.02 2.01 2.05 3/243 4035
/proc/meminfo: memFree=32018216/32770624 swapFree=5796/213976
[pid=3647] ppid=3644 vsize=52600 CPUtime=2142.11 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 43836 0 0 0 214209 2 0 0 20 0 1 0 931752 53862400 7260 33554432000 4194304 5970956 140724310328656 140724310325992 4977814 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7260 777 434 0 6514 0
Current children cumulated CPU time (s) 2142.11
Current children cumulated vsize (KiB) 52600

[startup+2202.31 s]
/proc/loadavg: 2.01 2.01 2.05 3/243 4109
/proc/meminfo: memFree=32014588/32770624 swapFree=5796/214100
[pid=3647] ppid=3644 vsize=52600 CPUtime=2202.11 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 43997 0 0 0 220209 2 0 0 20 0 1 0 931752 53862400 7260 33554432000 4194304 5970956 140724310328656 140724310325992 4977757 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7260 777 434 0 6514 0
Current children cumulated CPU time (s) 2202.11
Current children cumulated vsize (KiB) 52600

[startup+2262.3 s]
/proc/loadavg: 2.08 2.04 2.05 3/243 4110
/proc/meminfo: memFree=32013780/32770624 swapFree=5796/214176
[pid=3647] ppid=3644 vsize=52600 CPUtime=2262.1 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 44157 0 0 0 226208 2 0 0 20 0 1 0 931752 53862400 7260 33554432000 4194304 5970956 140724310328656 140724310325992 4930591 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7260 777 434 0 6514 0
Current children cumulated CPU time (s) 2262.1
Current children cumulated vsize (KiB) 52600

[startup+2322.3 s]
/proc/loadavg: 2.03 2.04 2.05 3/243 4111
/proc/meminfo: memFree=32013972/32770624 swapFree=5796/214244
[pid=3647] ppid=3644 vsize=52600 CPUtime=2322.11 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 44405 0 0 0 232208 3 0 0 20 0 1 0 931752 53862400 7260 33554432000 4194304 5970956 140724310328656 140724310325992 4976788 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7260 777 434 0 6514 0
Current children cumulated CPU time (s) 2322.11
Current children cumulated vsize (KiB) 52600

[startup+2382.3 s]
/proc/loadavg: 2.05 2.04 2.05 3/243 4112
/proc/meminfo: memFree=32013416/32770624 swapFree=5796/214300
[pid=3647] ppid=3644 vsize=52600 CPUtime=2382.1 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 44556 0 0 0 238207 3 0 0 20 0 1 0 931752 53862400 7260 33554432000 4194304 5970956 140724310328656 140724310325992 5098896 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7260 777 434 0 6514 0
Current children cumulated CPU time (s) 2382.1
Current children cumulated vsize (KiB) 52600



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+2400.2 s]
/proc/loadavg: 2.03 2.04 2.05 3/243 4112
/proc/meminfo: memFree=32013512/32770624 swapFree=5796/214316
[pid=3647] ppid=3644 vsize=52600 CPUtime=2400.01 cores=0,2,4,6,8,10,12,14
/proc/3647/stat : 3647 (cosoco) R 3644 3647 2448 0 -1 4202496 44705 0 0 0 239998 3 0 0 20 0 1 0 931752 53862400 7260 33554432000 4194304 5970956 140724310328656 140724310325992 4974141 0 0 4096 8405026 0 0 0 17 8 0 0 0 0 0 8070896 8075904 39141376 140724310336705 140724310336835 140724310336835 140724310339534 0
/proc/3647/statm: 13150 7260 777 434 0 6514 0
Current children cumulated CPU time (s) 2400.01
Current children cumulated vsize (KiB) 52600

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended.
??? exit watcher main loop
??? sigkill at end of watcher main loop
??? kill 3647 sig 9
??? kill -3647 sig 9
??? end of watcher thread
Dumping a history of the last processes samples

Child status: 1
Real time (s): 2400.21
CPU time (s): 2400.02
CPU user time (s): 2399.99
CPU system time (s): 0.030997
CPU usage (%): 99.992
Max. virtual memory (cumulated for all children) (KiB): 52600

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2399.99
system time used= 0.030997
maximum resident set size= 29084
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 44732
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 23
involuntary context switches= 120634

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 7.77074 second user time and 10.3703 second system time

The end

Launcher Data 

Begin job on node124 at 2017-07-21 19:04:00
IDJOB=4242020
IDBENCH=133181
IDSOLVER=2620
FILE ID=node124/4242020-1500656640
RUNJOBID= node124-1500648174-2481
PBS_JOBID= 20623296
Free space on /tmp= 62008 MiB

SOLVER NAME= cosoco 1.1
BENCH NAME= XCSP17/Ortholatin/Ortholatin-m1-s1/Ortholatin-024.xml
COMMAND LINE= DIR/cosoco -lc -model -restarts BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4242020-1500656640/watcher-4242020-1500656640 -o /tmp/evaluation-result-4242020-1500656640/solver-4242020-1500656640 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node124-1500648174-2481 --watchdog 2560  HOME/cosoco -lc -model -restarts HOME/instance-4242020-1500656640.xml

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 8

MD5SUM BENCH= 23eb83e679dec689804004fda8e6ed87
RANDOM SEED=2029947784

node124.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.16
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.16
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.16
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.16
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2133.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 4
apicid		: 1
initial apicid	: 1
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.16
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 4
apicid		: 17
initial apicid	: 17
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2133.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 4
apicid		: 3
initial apicid	: 3
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.16
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 4
apicid		: 19
initial apicid	: 19
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2267.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 4
apicid		: 5
initial apicid	: 5
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.16
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 4
apicid		: 21
initial apicid	: 21
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 4
apicid		: 7
initial apicid	: 7
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.16
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 4
apicid		: 23
initial apicid	: 23
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        32042876 kB
MemAvailable:   32028088 kB
Buffers:           31496 kB
Cached:           266396 kB
SwapCached:            0 kB
Active:           137344 kB
Inactive:         220796 kB
Active(anon):      63116 kB
Inactive(anon):     8704 kB
Active(file):      74228 kB
Inactive(file):   212092 kB
Unevictable:        5796 kB
Mlocked:            5796 kB
SwapTotal:      67108860 kB
SwapFree:       67108860 kB
Dirty:              2432 kB
Writeback:             0 kB
AnonPages:         68256 kB
Mapped:            30560 kB
Shmem:              8840 kB
Slab:              60860 kB
SReclaimable:      29456 kB
SUnreclaim:        31404 kB
KernelStack:        4224 kB
PageTables:         5148 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     330568 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     10240 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      112000 kB
DirectMap2M:    33433600 kB

Free space on /tmp at the end= 62004 MiB
End job on node124 at 2017-07-21 19:44:00