Trace number 4389563

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
cosoco 2? (TO) 2520.06 2520.01

General information on the benchmark

NameLatinSquare/LatinSquare-m1-gp/
qwh-o30-h374-11.xml
MD5SUM593ac311ff8adf8f7756cc1eeec69a70
Bench CategoryCSP (decision problem)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark18.7897
Satisfiable
(Un)Satisfiability was proved
Number of variables900
Number of constraints2
Number of domains1
Minimum domain size30
Maximum domain size30
Distribution of domain sizes[{"size":30,"count":900}]
Minimum variable degree1
Maximum variable degree2
Distribution of variable degrees[{"degree":1,"count":374},{"degree":2,"count":526}]
Minimum constraint arity526
Maximum constraint arity900
Distribution of constraint arities[{"arity":526,"count":1},{"arity":900,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"allDifferent","count":1},{"type":"instantiation","count":1}]
Optimization problemNO
Type of objective

Solver Data 

0.00/0.02	c root propagations     : 4768        
0.00/0.02	c restarts enabled, print stats at every restarts
0.00/0.02	c
0.00/0.02	c
0.00/0.02	c
0.00/0.02	c This is cosoco 2.00 --  
0.00/0.02	c
0.00/0.02	c command line: cosoco -lc -model -restarts -stick HOME/instance-4389563-1568358356.xml 
0.00/0.02	c ========================================[ Problem Statistics ]===========================================
0.00/0.02	c |                                                                                                       
0.00/0.02	c |  Parse time        :         0.00 s 
0.00/0.02	c |
0.00/0.02	c |               Variables: 900
0.00/0.02	c |            Domain Sizes: 30..30
0.00/0.02	c |
0.00/0.02	c |             Constraints: 586
0.00/0.02	c |                   Arity: 1..30  -- Unary: 526
0.00/0.02	c | 
0.00/0.02	c |                   Types: 
0.00/0.02	c |                          All Different: 60
0.00/0.02	c |                          Unary: 526
0.00/0.02	c |                          
0.00/0.02	c =========================================================================================================
0.00/0.02	c conflicts      decisions      filterCalls    propagations   Root Prop.     Nogoods        min/max/avg depth
0.00/0.07	c 101            963            12084          34674          882            0              576   623   590
0.10/0.13	c 211            2059           24017          83448          882            0              417   649   624
0.20/0.28	c 332            3943           50252          195919         882            0              335   663   621
0.40/0.44	c 465            5511           80112          338350         882            0              609   682   653
0.50/0.60	c 611            7360           109453         461550         882            0              436   724   680
0.69/0.73	c 771            8706           132589         564820         882            0              575   669   647
0.89/0.92	c 947            11138          166626         731575         882            0              604   731   688
0.99/1.04	c 1140           13069          188120         827573         882            0              596   722   683
1.19/1.21	c 1352           14902          219665         991697         882            0              582   722   691
1.39/1.41	c 1585           17504          256019         1169949        882            0              662   725   691
1.59/1.67	c 1841           19592          301784         1381330        882            0              601   667   629
1.89/2.00	c 2122           22415          359980         1652813        882            0              542   705   664
1.99/2.10	c 2431           23933          376842         1731712        882            0              550   568   557
2.19/2.22	c 2770           25789          398073         1821852        882            0              539   677   547
2.29/2.31	c 3142           27655          415609         1897096        882            0              531   711   557
2.59/2.69	c 3551           31230          483411         2212039        882            0              529   725   674
2.89/2.99	c 4000           33646          535569         2470443        882            0              531   685   647
3.09/3.12	c 4493           36199          560654         2584820        882            0              533   638   606
3.79/3.80	c 5035           41391          679503         3197121        882            0              537   724   692
4.29/4.34	c 5631           45217          774880         3669764        882            0              551   707   678
4.89/4.91	c 6286           50058          876746         4169118        882            0              552   724   687
5.49/5.58	c 7006           55743          995278         4785993        882            0              556   721   690
6.19/6.21	c 7798           61825          1107194        5377528        882            0              563   723   690
6.69/6.75	c 8669           67244          1204452        5809598        882            0              566   709   663
7.29/7.34	c 9627           73042          1309352        6343588        882            0              598   700   666
8.09/8.19	c 10680          80729          1459362        7049569        882            0              630   727   671
8.89/8.92	c 11838          88105          1590484        7706169        882            0              568   713   678
9.79/9.81	c 13111          95638          1748515        8508041        882            0              570   694   656
11.09/11.13	c 14511          108704         1983590        9644389        882            0              564   733   678
12.69/12.71	c 16051          119439         2264842        11073414       882            0              559   723   691
14.09/14.10	c 17745          131601         2512082        12405412       882            0              628   719   692
15.59/15.68	c 19608          145730         2794203        13801931       882            0              658   728   690
17.09/17.18	c 21657          157888         3056500        15338363       882            0              589   714   685
18.89/18.90	c 23910          170767         3358736        16829392       882            0              571   712   676
20.99/21.03	c 26388          187487         3737944        18710256       882            0              605   708   667
23.09/23.13	c 29113          204906         4110391        20773378       882            0              626   706   673
25.89/25.95	c 32110          230917         4613458        23283179       882            0              655   748   705
28.69/28.71	c 35406          256806         5108298        25809658       882            0              592   741   698
31.79/31.80	c 39031          281044         5657387        28537263       882            0              607   719   677
35.88/35.99	c 43018          318956         6403814        32310256       882            0              649   737   691
40.58/40.67	c 47403          355035         7231696        36658900       882            0              639   724   682
44.48/44.50	c 52226          386509         7903262        40465714       882            0              623   728   696
49.48/49.56	c 57531          419773         8801745        45225717       882            0              623   723   688
53.78/53.84	c 63366          453755         9564335        49467573       882            0              666   739   699
58.68/58.79	c 69784          499253         10454429       54033488       882            0              598   743   701
66.68/66.70	c 76843          559290         11851280       61398097       882            0              604   735   678
74.09/74.19	c 84607          608805         13163654       68274488       882            0              632   724   686
80.88/80.96	c 93147          671694         14378464       74163201       882            0              599   726   678
90.38/90.49	c 102541         742039         16068138       83023775       882            0              633   742   694
100.29/100.33	c 112874         824943         17814778       92000669       882            0              631   737   690
108.39/108.41	c 124240         898037         19245225       99692334       882            0              648   742   693
118.59/118.63	c 136742         983146         21058781       109543508      882            0              601   748   691
128.79/128.85	c 150494         1059715        22853757       119088326      882            0              599   720   680
141.49/141.57	c 165621         1165302        25127314       130666317      882            0              556   745   692
154.40/154.43	c 182260         1280490        27412138       142413594      882            0              583   733   692
167.60/167.65	c 200562         1395591        29755116       154332411      882            0              577   733   683
183.60/183.69	c 220694         1510898        32566489       170533369      882            0              563   732   689
199.50/199.54	c 242839         1670492        35383077       183741627      882            0              568   704   662
225.70/225.76	c 267198         1863670        40017822       207998006      882            0              575   756   693
257.50/257.50	c 293992         2062562        45619051       235768402      882            0              607   734   692
285.30/285.34	c 323465         2258423        50551610       260306056      882            0              646   732   689
313.01/313.02	c 355885         2465173        55477948       284103561      882            0              608   732   677
339.30/339.33	c 391547         2685774        60164785       307409960      882            0              650   732   690
377.80/377.89	c 430775         2975390        66986118       341668654      882            0              605   736   683
415.50/415.59	c 473925         3248552        73651147       375616916      882            0              590   727   681
459.11/459.16	c 521390         3570125        81367465       415470612      882            0              638   725   681
495.70/495.79	c 573601         3946811        87951708       446546210      882            0              607   732   686
535.60/535.61	c 631033         4323345        95099159       482016120      882            0              638   723   684
574.80/574.81	c 694208         4702330        102130974      516695377      882            0              642   737   695
627.30/627.31	c 763700         5136937        111500515      567567034      882            0              595   748   698
688.20/688.24	c 840141         5620253        122306909      621912432      882            0              633   730   682
758.81/758.80	c 924226         6191872        134796202      685961124      882            0              628   740   683
822.51/822.52	c 1016719        6748447        146100333      748062609      882            0              608   730   685
908.51/908.53	c 1118461        7427284        161337824      831036559      882            0              554   744   695
977.51/977.50	c 1230377        8095061        173772930      895521344      882            0              602   752   702
1054.21/1054.27	c 1353484        8751058        187467121      966148141      882            0              630   732   682
1143.02/1143.02	c 1488901        9474723        203188911      1047554601     882            0              624   744   688
1245.32/1245.33	c 1637859        10271668       221269264      1138117490     882            0              586   730   683
1343.72/1343.75	c 1801712        11145984       238692752      1228633291     882            0              614   727   684
1491.72/1491.79	c 1981950        12303473       264899117      1370703814     882            0              603   753   700
1658.53/1658.58	c 2180211        13447304       294200204      1526734205     882            0              597   729   685
1849.23/1849.20	c 2398298        14789174       327587586      1705947240     882            0              603   739   686
2068.23/2068.29	c 2638193        16318653       366034749      1900772280     882            0              616   724   675
2312.15/2312.16	c 2902077        18126767       409204224      2121414661     882            0              611   732   678
2520.05/2520.00	
2520.05/2520.00	c *** INTERRUPTED ***
2520.05/2520.00	c restarts              : 84          
2520.05/2520.00	c conflicts             : 3186628      (1265 /sec)
2520.05/2520.00	c decisions             : 19801581     (7858 /sec)
2520.05/2520.00	c propagations          : 2309768858  (916558 /sec)
2520.05/2520.00	c root propagations     : 4768        
2520.05/2520.00	c filter calls          : 446113794      (177026 /sec)
2520.05/2520.00	c useless filter calls  : 111619223      (25 %)
2520.05/2520.00	c Memory used           : 32.52 MB
2520.05/2520.00	c CPU time              : 2520.05 s
2520.05/2520.00	c real time             : 2520 s
2520.05/2520.00	
2520.05/2520.00	s UNKNOWN

Verifier Data


Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (15037 MiB free)
  memory of node 1: 16384 MiB (15501 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4389563-1568358356/watcher-4389563-1568358356 -o /tmp/evaluation-result-4389563-1568358356/solver-4389563-1568358356 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node151-1568353308-15004 --watchdog 2580 cosoco -lc -model -restarts -stick HOME/instance-4389563-1568358356.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=15719, runsolver pid=15716

[startup+0.100154 s]*
/proc/loadavg: 2.00 2.01 2.05 3/200 15720
/proc/meminfo: memFree=31266220/32770624 swapFree=66934164/67108860
[pid=15719] ppid=15716 vsize=32636 memory=9196 CPUtime=0.1 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 2471 0 0 0 10 0 0 0 20 0 1 0 412107110 33419264 2299 33554432000 4194304 6210612 140731265681744 140731265678280 140074156174309 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8159 2299 862 493 0 1464 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 32636 KiB
Current children cumulated memory: 9196 KiB

[startup+0.20966 s]*
/proc/loadavg: 2.00 2.01 2.05 3/200 15720
/proc/meminfo: memFree=31265848/32770624 swapFree=66934164/67108860
[pid=15719] ppid=15716 vsize=33032 memory=9460 CPUtime=0.2 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 2551 0 0 0 20 0 0 0 20 0 1 0 412107110 33824768 2365 33554432000 4194304 6210612 140731265681744 140731265678280 5192880 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8258 2365 862 493 0 1563 0
Current children cumulated CPU time: 0.2 s
Current children cumulated vsize: 33032 KiB
Current children cumulated memory: 9460 KiB

[startup+0.308962 s]*
/proc/loadavg: 2.00 2.01 2.05 3/200 15720
/proc/meminfo: memFree=31265848/32770624 swapFree=66934164/67108860
[pid=15719] ppid=15716 vsize=33032 memory=9460 CPUtime=0.3 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 2557 0 0 0 30 0 0 0 20 0 1 0 412107110 33824768 2365 33554432000 4194304 6210612 140731265681744 140731265678280 5074824 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8258 2365 862 493 0 1563 0
Current children cumulated CPU time: 0.3 s
Current children cumulated vsize: 33032 KiB
Current children cumulated memory: 9460 KiB

[startup+0.700277 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 15720
/proc/meminfo: memFree=31265848/32770624 swapFree=66934164/67108860
[pid=15719] ppid=15716 vsize=33032 memory=9460 CPUtime=0.69 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 2570 0 0 0 69 0 0 0 20 0 1 0 412107110 33824768 2365 33554432000 4194304 6210612 140731265681744 140731265679704 5074613 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8258 2365 862 493 0 1563 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 33032 KiB
Current children cumulated memory: 9460 KiB

[startup+1.50028 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 15720
/proc/meminfo: memFree=31265848/32770624 swapFree=66934164/67108860
[pid=15719] ppid=15716 vsize=33032 memory=9460 CPUtime=1.49 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 2573 0 0 0 149 0 0 0 20 0 1 0 412107110 33824768 2365 33554432000 4194304 6210612 140731265681744 140731265679704 5082684 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8258 2365 862 493 0 1563 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 33032 KiB
Current children cumulated memory: 9460 KiB

[startup+3.10899 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 15720
/proc/meminfo: memFree=31265848/32770624 swapFree=66934164/67108860
[pid=15719] ppid=15716 vsize=33032 memory=9652 CPUtime=3.09 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 4035 0 0 0 309 0 0 0 20 0 1 0 412107110 33824768 2413 33554432000 4194304 6210612 140731265681744 140731265679704 5074024 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8258 2413 862 493 0 1563 0
Current children cumulated CPU time: 3.09 s
Current children cumulated vsize: 33032 KiB
Current children cumulated memory: 9652 KiB

[startup+6.30026 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 15720
/proc/meminfo: memFree=31265848/32770624 swapFree=66934164/67108860
[pid=15719] ppid=15716 vsize=33164 memory=9680 CPUtime=6.29 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 5505 0 0 0 629 0 0 0 20 0 1 0 412107110 33959936 2420 33554432000 4194304 6210612 140731265681744 140731265679704 5397079 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8291 2420 862 493 0 1596 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 33164 KiB
Current children cumulated memory: 9680 KiB

[startup+12.7003 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 15720
/proc/meminfo: memFree=31265976/32770624 swapFree=66934164/67108860
[pid=15719] ppid=15716 vsize=33164 memory=9724 CPUtime=12.69 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 6988 0 0 0 1269 0 0 0 20 0 1 0 412107110 33959936 2431 33554432000 4194304 6210612 140731265681744 140731265679704 5078384 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8291 2431 862 493 0 1596 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 33164 KiB
Current children cumulated memory: 9724 KiB

[startup+25.5003 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 15720
/proc/meminfo: memFree=31266008/32770624 swapFree=66934164/67108860
[pid=15719] ppid=15716 vsize=33164 memory=9748 CPUtime=25.49 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 9943 0 0 0 2549 0 0 0 20 0 1 0 412107110 33959936 2437 33554432000 4194304 6210612 140731265681744 140731265679704 5074824 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8291 2437 862 493 0 1596 0
Current children cumulated CPU time: 25.49 s
Current children cumulated vsize: 33164 KiB
Current children cumulated memory: 9748 KiB

[startup+51.1092 s]
/proc/loadavg: 2.29 2.08 2.07 3/199 15787
/proc/meminfo: memFree=31273920/32770624 swapFree=66934164/67108860
[pid=15719] ppid=15716 vsize=33164 memory=9756 CPUtime=51.09 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 10412 0 0 0 5109 0 0 0 20 0 1 0 412107110 33959936 2439 33554432000 4194304 6210612 140731265681744 140731265679704 5078394 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8291 2439 862 493 0 1596 0
Current children cumulated CPU time: 51.09 s
Current children cumulated vsize: 33164 KiB
Current children cumulated memory: 9756 KiB

[startup+102.3 s]
/proc/loadavg: 2.20 2.10 2.07 3/199 15791
/proc/meminfo: memFree=31273872/32770624 swapFree=66934164/67108860
[pid=15719] ppid=15716 vsize=33164 memory=9764 CPUtime=102.29 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 10885 0 0 0 10229 0 0 0 20 0 1 0 412107110 33959936 2441 33554432000 4194304 6210612 140731265681744 140731265679704 5078583 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8291 2441 862 493 0 1596 0
Current children cumulated CPU time: 102.29 s
Current children cumulated vsize: 33164 KiB
Current children cumulated memory: 9764 KiB

[startup+162.3 s]
/proc/loadavg: 2.07 2.08 2.07 3/199 15795
/proc/meminfo: memFree=31273756/32770624 swapFree=66934164/67108860
[pid=15719] ppid=15716 vsize=33164 memory=9788 CPUtime=162.3 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 11359 0 0 0 16229 1 0 0 20 0 1 0 412107110 33959936 2447 33554432000 4194304 6210612 140731265681744 140731265679704 5081301 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8291 2447 862 493 0 1596 0
Current children cumulated CPU time: 162.3 s
Current children cumulated vsize: 33164 KiB
Current children cumulated memory: 9788 KiB

[startup+222.3 s]
/proc/loadavg: 2.03 2.06 2.06 3/199 15801
/proc/meminfo: memFree=31272948/32770624 swapFree=66934164/67108860
[pid=15719] ppid=15716 vsize=33296 memory=9796 CPUtime=222.3 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 11836 0 0 0 22229 1 0 0 20 0 1 0 412107110 34095104 2449 33554432000 4194304 6210612 140731265681744 140731265679336 5078504 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8324 2449 862 493 0 1629 0
Current children cumulated CPU time: 222.3 s
Current children cumulated vsize: 33296 KiB
Current children cumulated memory: 9796 KiB

[startup+282.3 s]
/proc/loadavg: 2.06 2.07 2.06 3/199 15805
/proc/meminfo: memFree=31272688/32770624 swapFree=66934164/67108860
[pid=15719] ppid=15716 vsize=33296 memory=9800 CPUtime=282.3 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 12307 0 0 0 28229 1 0 0 20 0 1 0 412107110 34095104 2450 33554432000 4194304 6210612 140731265681744 140731265679704 5082435 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8324 2450 862 493 0 1629 0
Current children cumulated CPU time: 282.3 s
Current children cumulated vsize: 33296 KiB
Current children cumulated memory: 9800 KiB

[startup+342.3 s]
/proc/loadavg: 2.47 2.17 2.09 3/200 15817
/proc/meminfo: memFree=31272632/32770624 swapFree=66934168/67108860
[pid=15719] ppid=15716 vsize=33296 memory=9800 CPUtime=342.3 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 12782 0 0 0 34229 1 0 0 20 0 1 0 412107110 34095104 2450 33554432000 4194304 6210612 140731265681744 140731265679704 5081338 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8324 2450 862 493 0 1629 0
Current children cumulated CPU time: 342.3 s
Current children cumulated vsize: 33296 KiB
Current children cumulated memory: 9800 KiB

[startup+402.3 s]
/proc/loadavg: 2.24 2.16 2.10 3/200 15823
/proc/meminfo: memFree=31272144/32770624 swapFree=66934168/67108860
[pid=15719] ppid=15716 vsize=33296 memory=9804 CPUtime=402.3 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 13258 0 0 0 40229 1 0 0 20 0 1 0 412107110 34095104 2451 33554432000 4194304 6210612 140731265681744 140731265679704 5075666 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8324 2451 862 493 0 1629 0
Current children cumulated CPU time: 402.3 s
Current children cumulated vsize: 33296 KiB
Current children cumulated memory: 9804 KiB

[startup+462.3 s]
/proc/loadavg: 2.09 2.13 2.09 3/200 15827
/proc/meminfo: memFree=31272332/32770624 swapFree=66934168/67108860
[pid=15719] ppid=15716 vsize=33296 memory=9808 CPUtime=462.3 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 13731 0 0 0 46229 1 0 0 20 0 1 0 412107110 34095104 2452 33554432000 4194304 6210612 140731265681744 140731265679704 5081875 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8324 2452 862 493 0 1629 0
Current children cumulated CPU time: 462.3 s
Current children cumulated vsize: 33296 KiB
Current children cumulated memory: 9808 KiB


################
# More data... #
################

/proc/15719/statm: 8324 2453 862 493 0 1629 0
Current children cumulated CPU time: 1722.33 s
Current children cumulated vsize: 33296 KiB
Current children cumulated memory: 9812 KiB

[startup+1782.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 15967
/proc/meminfo: memFree=31271496/32770624 swapFree=66934180/67108860
[pid=15719] ppid=15716 vsize=33296 memory=9812 CPUtime=1782.33 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 22826 0 0 0 178232 1 0 0 20 0 1 0 412107110 34095104 2453 33554432000 4194304 6210612 140731265681744 140731265679704 5074869 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8324 2453 862 493 0 1629 0
Current children cumulated CPU time: 1782.33 s
Current children cumulated vsize: 33296 KiB
Current children cumulated memory: 9812 KiB

[startup+1842.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 15973
/proc/meminfo: memFree=31271592/32770624 swapFree=66934180/67108860
[pid=15719] ppid=15716 vsize=33296 memory=9812 CPUtime=1842.33 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 23130 0 0 0 184232 1 0 0 20 0 1 0 412107110 34095104 2453 33554432000 4194304 6210612 140731265681744 140731265679704 5074901 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8324 2453 862 493 0 1629 0
Current children cumulated CPU time: 1842.33 s
Current children cumulated vsize: 33296 KiB
Current children cumulated memory: 9812 KiB

[startup+1902.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 15977
/proc/meminfo: memFree=31271096/32770624 swapFree=66934180/67108860
[pid=15719] ppid=15716 vsize=33296 memory=9812 CPUtime=1902.33 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 23609 0 0 0 190232 1 0 0 20 0 1 0 412107110 34095104 2453 33554432000 4194304 6210612 140731265681744 140731265679704 5080952 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8324 2453 862 493 0 1629 0
Current children cumulated CPU time: 1902.33 s
Current children cumulated vsize: 33296 KiB
Current children cumulated memory: 9812 KiB

[startup+1962.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 15981
/proc/meminfo: memFree=31271460/32770624 swapFree=66934180/67108860
[pid=15719] ppid=15716 vsize=33296 memory=9812 CPUtime=1962.33 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 23914 0 0 0 196232 1 0 0 20 0 1 0 412107110 34095104 2453 33554432000 4194304 6210612 140731265681744 140731265679704 5076034 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8324 2453 862 493 0 1629 0
Current children cumulated CPU time: 1962.33 s
Current children cumulated vsize: 33296 KiB
Current children cumulated memory: 9812 KiB

[startup+2022.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 15987
/proc/meminfo: memFree=31271384/32770624 swapFree=66934180/67108860
[pid=15719] ppid=15716 vsize=33296 memory=9812 CPUtime=2022.33 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 24221 0 0 0 202232 1 0 0 20 0 1 0 412107110 34095104 2453 33554432000 4194304 6210612 140731265681744 140731265679704 5074866 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8324 2453 862 493 0 1629 0
Current children cumulated CPU time: 2022.33 s
Current children cumulated vsize: 33296 KiB
Current children cumulated memory: 9812 KiB

[startup+2082.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 15991
/proc/meminfo: memFree=31271728/32770624 swapFree=66934180/67108860
[pid=15719] ppid=15716 vsize=33296 memory=9812 CPUtime=2082.33 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 24698 0 0 0 208232 1 0 0 20 0 1 0 412107110 34095104 2453 33554432000 4194304 6210612 140731265681744 140731265679704 5074212 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8324 2453 862 493 0 1629 0
Current children cumulated CPU time: 2082.33 s
Current children cumulated vsize: 33296 KiB
Current children cumulated memory: 9812 KiB

[startup+2142.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 16003
/proc/meminfo: memFree=31271816/32770624 swapFree=66934180/67108860
[pid=15719] ppid=15716 vsize=33296 memory=9812 CPUtime=2142.33 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 25001 0 0 0 214232 1 0 0 20 0 1 0 412107110 34095104 2453 33554432000 4194304 6210612 140731265681744 140731265679704 5075203 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8324 2453 862 493 0 1629 0
Current children cumulated CPU time: 2142.33 s
Current children cumulated vsize: 33296 KiB
Current children cumulated memory: 9812 KiB

[startup+2202.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 16009
/proc/meminfo: memFree=31270940/32770624 swapFree=66934180/67108860
[pid=15719] ppid=15716 vsize=33296 memory=9812 CPUtime=2202.34 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 25307 0 0 0 220233 1 0 0 20 0 1 0 412107110 34095104 2453 33554432000 4194304 6210612 140731265681744 140731265679704 5081999 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8324 2453 862 493 0 1629 0
Current children cumulated CPU time: 2202.34 s
Current children cumulated vsize: 33296 KiB
Current children cumulated memory: 9812 KiB

[startup+2262.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 16013
/proc/meminfo: memFree=31271860/32770624 swapFree=66934180/67108860
[pid=15719] ppid=15716 vsize=33296 memory=9812 CPUtime=2262.34 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 25613 0 0 0 226233 1 0 0 20 0 1 0 412107110 34095104 2453 33554432000 4194304 6210612 140731265681744 140731265679704 5076021 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8324 2453 862 493 0 1629 0
Current children cumulated CPU time: 2262.34 s
Current children cumulated vsize: 33296 KiB
Current children cumulated memory: 9812 KiB

[startup+2322.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 16017
/proc/meminfo: memFree=31271180/32770624 swapFree=66934180/67108860
[pid=15719] ppid=15716 vsize=33296 memory=9812 CPUtime=2322.34 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 26089 0 0 0 232233 1 0 0 20 0 1 0 412107110 34095104 2453 33554432000 4194304 6210612 140731265681744 140731265679704 5074901 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8324 2453 862 493 0 1629 0
Current children cumulated CPU time: 2322.34 s
Current children cumulated vsize: 33296 KiB
Current children cumulated memory: 9812 KiB

[startup+2382.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 16023
/proc/meminfo: memFree=31271380/32770624 swapFree=66934180/67108860
[pid=15719] ppid=15716 vsize=33296 memory=9812 CPUtime=2382.34 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 26398 0 0 0 238233 1 0 0 20 0 1 0 412107110 34095104 2453 33554432000 4194304 6210612 140731265681744 140731265679704 5080772 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8324 2453 862 493 0 1629 0
Current children cumulated CPU time: 2382.34 s
Current children cumulated vsize: 33296 KiB
Current children cumulated memory: 9812 KiB

[startup+2442.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 16027
/proc/meminfo: memFree=31271572/32770624 swapFree=66934180/67108860
[pid=15719] ppid=15716 vsize=33296 memory=9812 CPUtime=2442.34 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 26707 0 0 0 244233 1 0 0 20 0 1 0 412107110 34095104 2453 33554432000 4194304 6210612 140731265681744 140731265679704 5409279 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8324 2453 862 493 0 1629 0
Current children cumulated CPU time: 2442.34 s
Current children cumulated vsize: 33296 KiB
Current children cumulated memory: 9812 KiB

[startup+2502.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 16031
/proc/meminfo: memFree=31271572/32770624 swapFree=66934180/67108860
[pid=15719] ppid=15716 vsize=33296 memory=9812 CPUtime=2502.34 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 27016 0 0 0 250233 1 0 0 20 0 1 0 412107110 34095104 2453 33554432000 4194304 6210612 140731265681744 140731265679704 5081610 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8324 2453 862 493 0 1629 0
Current children cumulated CPU time: 2502.34 s
Current children cumulated vsize: 33296 KiB
Current children cumulated memory: 9812 KiB



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+2520 s]*
/proc/loadavg: 2.00 2.01 2.05 3/199 16034
/proc/meminfo: memFree=31271200/32770624 swapFree=66934180/67108860
[pid=15719] ppid=15716 vsize=33296 memory=9812 CPUtime=2520.05 cores=1,3,5,7
/proc/15719/stat : 15719 (cosoco) R 15716 15719 14972 0 -1 4202496 27321 0 0 0 252004 1 0 0 20 0 1 0 412107110 34095104 2453 33554432000 4194304 6210612 140731265681744 140731265679704 5081875 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 9011200 140731265688587 140731265688689 140731265688689 140731265691598 0
/proc/15719/statm: 8324 2453 862 493 0 1629 0
Current children cumulated CPU time: 2520.05 s
Current children cumulated vsize: 33296 KiB
Current children cumulated memory: 9812 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

[startup+2520.01 s]
# the end of solver process 15719 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2520.05 s, system=0.013 s

Solver just ended.
??? end of watcher thread
Dumping a history of the last processes samples

Child status: 1
Real time (s): 2520.01
CPU time (s): 2520.06
CPU user time (s): 2520.05
CPU system time (s): 0.013
CPU usage (%): 100.002
Max. virtual memory (cumulated for all children) (KiB): 33296
Max. memory (cumulated for all children) (KiB): 9812

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2520.05
system time used= 0.013
maximum resident set size= 9880
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 27376
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 16
involuntary context switches= 666


# summary of solver processes directly reported to runsolver:
#   pid: 15719
#   total CPU time (s): 2520.06
#   total CPU user time (s): 2520.05
#   total CPU system time (s): 0.013

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 8.48019 second user time and 16.3869 second system time

The end

Launcher Data 

Begin job on node151 at 2019-09-13 09:05:17
IDJOB=4389563
IDBENCH=138937
IDSOLVER=2829
FILE ID=node151/4389563-1568358356
RUNJOBID= node151-1568353308-15004
PBS_JOBID= 21701185
Free space on /tmp= 56996 MiB

SOLVER NAME= cosoco 2
BENCH NAME= XCSP17/LatinSquare/LatinSquare-m1-gp/qwh-o30-h374-11.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4389563-1568358356/watcher-4389563-1568358356 -o /tmp/evaluation-result-4389563-1568358356/solver-4389563-1568358356 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node151-1568353308-15004 --watchdog 2580  cosoco -lc -model -restarts -stick HOME/instance-4389563-1568358356.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 593ac311ff8adf8f7756cc1eeec69a70
RANDOM SEED=1758502985

node151.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.53
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5335.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.53
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5335.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.53
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5335.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.53
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5335.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        31272468 kB
MemAvailable:   31842788 kB
Buffers:          146756 kB
Cached:           714796 kB
SwapCached:        72052 kB
Active:           448652 kB
Inactive:         500940 kB
Active(anon):      23580 kB
Inactive(anon):    96488 kB
Active(file):     425072 kB
Inactive(file):   404452 kB
Unevictable:      111208 kB
Mlocked:          111208 kB
SwapTotal:      67108860 kB
SwapFree:       66934164 kB
Dirty:              2796 kB
Writeback:             0 kB
AnonPages:        141604 kB
Mapped:            37424 kB
Shmem:             23412 kB
Slab:             141324 kB
SReclaimable:     113264 kB
SUnreclaim:        28060 kB
KernelStack:        3776 kB
PageTables:         5212 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     537628 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     96256 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      138624 kB
DirectMap2M:    33406976 kB

Free space on /tmp at the end= 56996 MiB
End job on node151 at 2019-09-13 09:47:18