Trace number 4389570

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
cosoco 2? (TO) 2520.08 2520.01

General information on the benchmark

NameLatinSquare/LatinSquare-m1-gp/
qwh-o30-h375-20.xml
MD5SUMcc39eb921a5cd1c2fcd1d541ff73edd4
Bench CategoryCSP (decision problem)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark28.8683
Satisfiable
(Un)Satisfiability was proved
Number of variables900
Number of constraints2
Number of domains1
Minimum domain size30
Maximum domain size30
Distribution of domain sizes[{"size":30,"count":900}]
Minimum variable degree1
Maximum variable degree2
Distribution of variable degrees[{"degree":1,"count":375},{"degree":2,"count":525}]
Minimum constraint arity525
Maximum constraint arity900
Distribution of constraint arities[{"arity":525,"count":1},{"arity":900,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"allDifferent","count":1},{"type":"instantiation","count":1}]
Optimization problemNO
Type of objective

Solver Data 

0.00/0.02	c root propagations     : 4641        
0.00/0.02	c restarts enabled, print stats at every restarts
0.00/0.02	c
0.00/0.02	c
0.00/0.02	c
0.00/0.02	c This is cosoco 2.00 --  
0.00/0.02	c
0.00/0.02	c command line: cosoco -lc -model -restarts -stick HOME/instance-4389570-1568358389.xml 
0.00/0.02	c ========================================[ Problem Statistics ]===========================================
0.00/0.02	c |                                                                                                       
0.00/0.02	c |  Parse time        :         0.01 s 
0.00/0.02	c |
0.00/0.02	c |               Variables: 900
0.00/0.02	c |            Domain Sizes: 30..30
0.00/0.02	c |
0.00/0.02	c |             Constraints: 585
0.00/0.02	c |                   Arity: 1..30  -- Unary: 525
0.00/0.02	c | 
0.00/0.02	c |                   Types: 
0.00/0.02	c |                          All Different: 60
0.00/0.02	c |                          Unary: 525
0.00/0.02	c |                          
0.00/0.02	c =========================================================================================================
0.00/0.02	c conflicts      decisions      filterCalls    propagations   Root Prop.     Nogoods        min/max/avg depth
0.00/0.10	c 101            1245           17779          61708          889            0              613   674   645
0.10/0.16	c 211            2335           28342          107839         889            0              480   559   528
0.20/0.24	c 332            3526           42423          168565         889            0              433   588   555
0.20/0.29	c 465            4519           51614          212771         889            0              423   507   490
0.30/0.40	c 611            6522           74135          298078         889            0              399   604   564
0.40/0.48	c 771            7882           87547          355488         889            0              470   620   568
0.59/0.65	c 947            10176          118766         488531         889            0              502   695   640
0.68/0.73	c 1140           11719          133703         544086         889            0              554   662   632
0.88/0.90	c 1352           13580          165097         710537         889            0              497   703   678
0.99/1.09	c 1585           15780          198435         878280         889            0              618   705   678
1.28/1.31	c 1841           17813          238820         1081512        889            0              535   712   685
1.48/1.51	c 2122           20035          274655         1262902        889            0              546   730   687
1.58/1.67	c 2431           21892          302430         1391929        889            0              527   745   601
1.78/1.82	c 2770           24105          330152         1511256        889            0              593   708   679
1.99/2.05	c 3142           26611          372101         1720976        889            0              585   685   658
2.28/2.36	c 3551           29599          428234         2010001        889            0              622   723   691
2.78/2.82	c 4000           34447          511053         2433237        889            0              667   736   704
3.18/3.23	c 4493           39151          585521         2756778        889            0              643   701   675
3.58/3.64	c 5035           43403          659535         3132079        889            0              559   730   693
4.18/4.21	c 5631           49409          763579         3577292        889            0              575   728   694
4.88/4.92	c 6286           55245          890554         4189065        889            0              642   729   687
5.78/5.81	c 7006           59860          1048030        5017824        889            0              573   693   662
6.48/6.50	c 7798           65008          1171804        5641299        889            0              591   717   684
7.38/7.41	c 8669           72620          1332518        6418275        889            0              595   715   671
8.09/8.16	c 9627           80067          1470609        7036393        889            0              613   723   687
8.78/8.81	c 10680          85191          1584938        7595860        889            0              627   706   679
9.78/9.85	c 11838          94613          1773516        8491247        889            0              554   730   689
10.68/10.72	c 13111          102541         1929483        9221344        889            0              556   711   682
11.78/11.85	c 14511          112174         2132243        10173149       889            0              544   712   676
12.78/12.87	c 16051          120147         2312162        11071218       889            0              570   690   647
14.28/14.32	c 17745          131505         2570966        12358518       889            0              569   693   659
15.78/15.86	c 19608          145119         2847117        13624156       889            0              571   706   664
17.28/17.33	c 21657          158450         3105945        14953229       889            0              581   708   670
18.58/18.65	c 23910          168638         3343799        16129572       889            0              581   689   668
19.99/20.09	c 26388          181995         3600513        17362050       889            0              588   701   658
21.68/21.77	c 29113          197156         3903024        18859572       889            0              580   714   674
24.18/24.21	c 32110          215858         4341355        20946850       889            0              580   725   693
26.38/26.40	c 35406          235474         4735113        22899132       889            0              593   730   697
29.18/29.29	c 39031          258014         5249661        25546916       889            0              638   719   677
32.18/32.28	c 43018          283669         5781878        28178542       889            0              629   727   675
35.18/35.25	c 47403          316836         6315123        30818654       889            0              630   720   678
37.68/37.71	c 52226          340665         6751362        33102629       889            0              573   706   667
40.58/40.66	c 57531          363201         7276954        35674784       889            0              580   698   660
46.28/46.37	c 63366          400946         8291207        40578899       889            0              564   726   685
51.78/51.87	c 69784          452833         9277351        45313096       889            0              580   713   670
61.48/61.52	c 76843          520118         10982704       53272313       889            0              567   704   660
66.99/67.07	c 84607          573439         11972001       58165545       889            0              571   721   669
72.99/73.07	c 93147          619963         13045850       63743446       889            0              562   721   676
79.78/79.88	c 102541         671559         14242845       69678223       889            0              568   717   687
86.38/86.48	c 112874         735325         15408921       75505313       889            0              555   705   658
96.99/97.06	c 124240         811926         17287579       83932974       889            0              586   706   651
104.48/104.57	c 136742         878163         18638317       90785460       889            0              548   716   671
111.18/111.27	c 150494         958493         19842789       95940404       889            0              540   698   642
121.09/121.16	c 165621         1034345        21606739       104560192      889            0              539   701   667
133.89/133.90	c 182260         1121479        23853174       115892714      889            0              541   686   652
147.99/148.06	c 200562         1229407        26371868       128243609      889            0              585   725   666
164.89/164.98	c 220694         1345395        29356186       141214786      889            0              544   702   650
173.59/173.62	c 242839         1466742        30943015       148286269      889            0              547   718   654
189.00/189.01	c 267198         1596253        33680448       161307368      889            0              557   706   651
209.39/209.40	c 293992         1767496        37293138       178944938      889            0              562   712   658
226.39/226.46	c 323465         1912775        40316274       193152994      889            0              540   706   651
249.19/249.25	c 355885         2096907        44369500       213118928      889            0              560   724   673
274.59/274.62	c 391547         2327002        48882178       234580339      889            0              541   719   654
299.19/299.27	c 430775         2545851        53282033       256005888      889            0              557   728   673
327.29/327.33	c 473925         2800527        58288614       279973656      889            0              540   729   671
360.89/360.97	c 521390         3136179        64311265       307782651      889            0              547   721   657
398.29/398.36	c 573601         3531889        71034446       339802659      889            0              559   735   678
449.80/449.87	c 631033         3920330        80154578       383228806      889            0              567   708   648
496.40/496.44	c 694208         4287497        88470856       423663385      889            0              547   748   687
531.11/531.14	c 763700         4722373        94746510       451126845      889            0              539   726   648
568.30/568.35	c 840141         5125639        101443582      479938831      889            0              537   723   653
628.60/628.62	c 924226         5765995        112157744      535772750      889            0              545   723   675
675.80/675.82	c 1016719        6267062        120816688      572192031      889            0              547   719   651
746.90/746.97	c 1118461        7004110        133564241      629408590      889            0              546   723   655
831.51/831.50	c 1230377        7775237        148701588      697636568      889            0              547   713   663
911.41/911.45	c 1353484        8386105        162855617      768098719      889            0              552   693   649
1004.71/1004.77	c 1488901        9138397        179369086      849987727      889            0              549   703   650
1107.02/1107.04	c 1637859        9923915        197460327      946760209      889            0              552   718   670
1209.12/1209.14	c 1801712        10839118       215680537      1039534013     889            0              566   713   661
1327.62/1327.65	c 1981950        11964080       236892472      1144010616     889            0              556   747   689
1483.32/1483.30	c 2180211        13116245       264534931      1280704826     889            0              554   720   670
1666.64/1666.64	c 2398298        14324908       296814640      1450136529     889            0              558   729   683
1905.25/1905.27	c 2638193        15873903       338913999      1670033720     889            0              578   745   689
2113.55/2113.57	c 2902077        17433276       376035002      1859569030     889            0              619   745   685
2342.56/2342.58	c 3192349        19217918       416860786      2071570326     889            0              596   747   684
2520.07/2520.00	
2520.07/2520.00	c *** INTERRUPTED ***
2520.07/2520.00	c restarts              : 85          
2520.07/2520.00	c conflicts             : 3425216      (1359 /sec)
2520.07/2520.00	c decisions             : 20509534     (8138 /sec)
2520.07/2520.00	c propagations          : 2236222504  (887367 /sec)
2520.07/2520.00	c root propagations     : 4641        
2520.07/2520.00	c filter calls          : 448506514      (177974 /sec)
2520.07/2520.00	c useless filter calls  : 114935924      (25 %)
2520.07/2520.00	c Memory used           : 32.52 MB
2520.07/2520.00	c CPU time              : 2520.07 s
2520.07/2520.00	c real time             : 2520 s
2520.07/2520.00	
2520.07/2520.00	s UNKNOWN

Verifier Data


Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (15052 MiB free)
  memory of node 1: 16384 MiB (15494 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4389570-1568358389/watcher-4389570-1568358389 -o /tmp/evaluation-result-4389570-1568358389/solver-4389570-1568358389 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node151-1568353308-15003 --watchdog 2580 cosoco -lc -model -restarts -stick HOME/instance-4389570-1568358389.xml 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=15785, runsolver pid=15782
Current StackSize limit: 8192 KiB


[startup+0.100861 s]*
/proc/loadavg: 2.00 2.01 2.05 3/200 15786
/proc/meminfo: memFree=31275492/32770624 swapFree=66934164/67108860
[pid=15785] ppid=15782 vsize=32504 memory=8936 CPUtime=0.1 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 2425 0 0 0 10 0 0 0 20 0 1 0 412110305 33284096 2234 33554432000 4194304 6210612 140733409275952 140733409273240 5081394 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8126 2234 859 493 0 1431 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 32504 KiB
Current children cumulated memory: 8936 KiB

[startup+0.209756 s]*
/proc/loadavg: 2.00 2.01 2.05 3/200 15786
/proc/meminfo: memFree=31275492/32770624 swapFree=66934164/67108860
[pid=15785] ppid=15782 vsize=32768 memory=9200 CPUtime=0.2 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 2488 0 0 0 20 0 0 0 20 0 1 0 412110305 33554432 2300 33554432000 4194304 6210612 140733409275952 140733409273544 140011610160101 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8192 2300 862 493 0 1497 0
Current children cumulated CPU time: 0.2 s
Current children cumulated vsize: 32768 KiB
Current children cumulated memory: 9200 KiB

[startup+0.309137 s]*
/proc/loadavg: 2.00 2.01 2.05 3/200 15786
/proc/meminfo: memFree=31275492/32770624 swapFree=66934164/67108860
[pid=15785] ppid=15782 vsize=32768 memory=9200 CPUtime=0.3 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 2505 0 0 0 30 0 0 0 20 0 1 0 412110305 33554432 2300 33554432000 4194304 6210612 140733409275952 140733409273544 5081007 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8192 2300 862 493 0 1497 0
Current children cumulated CPU time: 0.3 s
Current children cumulated vsize: 32768 KiB
Current children cumulated memory: 9200 KiB

[startup+0.70026 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 15786
/proc/meminfo: memFree=31275168/32770624 swapFree=66934164/67108860
[pid=15785] ppid=15782 vsize=33032 memory=9464 CPUtime=0.68 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 2575 0 0 0 68 0 0 0 20 0 1 0 412110305 33824768 2366 33554432000 4194304 6210612 140733409275952 140733409273912 5078394 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8258 2366 862 493 0 1563 0
Current children cumulated CPU time: 0.68 s
Current children cumulated vsize: 33032 KiB
Current children cumulated memory: 9464 KiB

[startup+1.50027 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 15786
/proc/meminfo: memFree=31275372/32770624 swapFree=66934164/67108860
[pid=15785] ppid=15782 vsize=33168 memory=9464 CPUtime=1.48 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 2586 0 0 0 148 0 0 0 20 0 1 0 412110305 33964032 2366 33554432000 4194304 6210612 140733409275952 140733409273912 5078678 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8292 2366 862 493 0 1597 0
Current children cumulated CPU time: 1.48 s
Current children cumulated vsize: 33168 KiB
Current children cumulated memory: 9464 KiB

[startup+3.1093 s]
/proc/loadavg: 2.08 2.03 2.05 3/200 15786
/proc/meminfo: memFree=31273864/32770624 swapFree=66934164/67108860
[pid=15785] ppid=15782 vsize=33168 memory=9708 CPUtime=3.09 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 3048 0 0 0 309 0 0 0 20 0 1 0 412110305 33964032 2427 33554432000 4194304 6210612 140733409275952 140733409273912 5078899 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8292 2427 862 493 0 1597 0
Current children cumulated CPU time: 3.09 s
Current children cumulated vsize: 33168 KiB
Current children cumulated memory: 9708 KiB

[startup+6.30026 s]
/proc/loadavg: 2.08 2.03 2.05 3/200 15786
/proc/meminfo: memFree=31273988/32770624 swapFree=66934164/67108860
[pid=15785] ppid=15782 vsize=33168 memory=9716 CPUtime=6.28 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 3507 0 0 0 628 0 0 0 20 0 1 0 412110305 33964032 2429 33554432000 4194304 6210612 140733409275952 140733409273912 5078394 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8292 2429 862 493 0 1597 0
Current children cumulated CPU time: 6.28 s
Current children cumulated vsize: 33168 KiB
Current children cumulated memory: 9716 KiB

[startup+12.7003 s]
/proc/loadavg: 2.22 2.06 2.06 3/200 15787
/proc/meminfo: memFree=31274244/32770624 swapFree=66934164/67108860
[pid=15785] ppid=15782 vsize=33168 memory=9732 CPUtime=12.68 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 3973 0 0 0 1268 0 0 0 20 0 1 0 412110305 33964032 2433 33554432000 4194304 6210612 140733409275952 140733409273912 5074348 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8292 2433 862 493 0 1597 0
Current children cumulated CPU time: 12.68 s
Current children cumulated vsize: 33168 KiB
Current children cumulated memory: 9732 KiB

[startup+25.5003 s]
/proc/loadavg: 2.42 2.11 2.08 3/199 15787
/proc/meminfo: memFree=31274284/32770624 swapFree=66934164/67108860
[pid=15785] ppid=15782 vsize=33168 memory=9800 CPUtime=25.48 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 4926 0 0 0 2548 0 0 0 20 0 1 0 412110305 33964032 2450 33554432000 4194304 6210612 140733409275952 140733409273912 5075650 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8292 2450 862 493 0 1597 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 33168 KiB
Current children cumulated memory: 9800 KiB

[startup+51.1092 s]
/proc/loadavg: 2.28 2.10 2.07 3/199 15789
/proc/meminfo: memFree=31274056/32770624 swapFree=66934164/67108860
[pid=15785] ppid=15782 vsize=33300 memory=9816 CPUtime=51.09 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 5415 0 0 0 5109 0 0 0 20 0 1 0 412110305 34099200 2454 33554432000 4194304 6210612 140733409275952 140733409273912 5074172 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8325 2454 862 493 0 1630 0
Current children cumulated CPU time: 51.09 s
Current children cumulated vsize: 33300 KiB
Current children cumulated memory: 9816 KiB

[startup+102.3 s]
/proc/loadavg: 2.11 2.08 2.07 3/199 15793
/proc/meminfo: memFree=31273508/32770624 swapFree=66934164/67108860
[pid=15785] ppid=15782 vsize=33300 memory=9828 CPUtime=102.28 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 5898 0 0 0 10228 0 0 0 20 0 1 0 412110305 34099200 2457 33554432000 4194304 6210612 140733409275952 140733409273912 5082063 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8325 2457 862 493 0 1630 0
Current children cumulated CPU time: 102.28 s
Current children cumulated vsize: 33300 KiB
Current children cumulated memory: 9828 KiB

[startup+162.3 s]
/proc/loadavg: 2.04 2.07 2.06 3/199 15797
/proc/meminfo: memFree=31273532/32770624 swapFree=66934164/67108860
[pid=15785] ppid=15782 vsize=33300 memory=9828 CPUtime=162.29 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 6383 0 0 0 16229 0 0 0 20 0 1 0 412110305 34099200 2457 33554432000 4194304 6210612 140733409275952 140733409273912 5081990 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8325 2457 862 493 0 1630 0
Current children cumulated CPU time: 162.29 s
Current children cumulated vsize: 33300 KiB
Current children cumulated memory: 9828 KiB

[startup+222.3 s]
/proc/loadavg: 2.10 2.07 2.06 3/200 15803
/proc/meminfo: memFree=31272560/32770624 swapFree=66934164/67108860
[pid=15785] ppid=15782 vsize=33300 memory=9840 CPUtime=222.29 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 6871 0 0 0 22229 0 0 0 20 0 1 0 412110305 34099200 2460 33554432000 4194304 6210612 140733409275952 140733409273912 5078394 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8325 2460 862 493 0 1630 0
Current children cumulated CPU time: 222.29 s
Current children cumulated vsize: 33300 KiB
Current children cumulated memory: 9840 KiB

[startup+282.3 s]
/proc/loadavg: 2.19 2.09 2.07 3/200 15815
/proc/meminfo: memFree=31272384/32770624 swapFree=66934168/67108860
[pid=15785] ppid=15782 vsize=33300 memory=9844 CPUtime=282.29 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 7359 0 0 0 28229 0 0 0 20 0 1 0 412110305 34099200 2461 33554432000 4194304 6210612 140733409275952 140733409273912 5078927 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8325 2461 862 493 0 1630 0
Current children cumulated CPU time: 282.29 s
Current children cumulated vsize: 33300 KiB
Current children cumulated memory: 9844 KiB

[startup+342.3 s]
/proc/loadavg: 2.36 2.18 2.10 3/200 15819
/proc/meminfo: memFree=31272224/32770624 swapFree=66934168/67108860
[pid=15785] ppid=15782 vsize=33300 memory=9848 CPUtime=342.29 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 7846 0 0 0 34229 0 0 0 20 0 1 0 412110305 34099200 2462 33554432000 4194304 6210612 140733409275952 140733409273912 5082025 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8325 2462 862 493 0 1630 0
Current children cumulated CPU time: 342.29 s
Current children cumulated vsize: 33300 KiB
Current children cumulated memory: 9848 KiB

[startup+402.3 s]
/proc/loadavg: 2.13 2.15 2.09 3/201 15825
/proc/meminfo: memFree=31272128/32770624 swapFree=66934168/67108860
[pid=15785] ppid=15782 vsize=33300 memory=9848 CPUtime=402.29 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 8335 0 0 0 40229 0 0 0 20 0 1 0 412110305 34099200 2462 33554432000 4194304 6210612 140733409275952 140733409273912 5074853 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8325 2462 862 493 0 1630 0
Current children cumulated CPU time: 402.29 s
Current children cumulated vsize: 33300 KiB
Current children cumulated memory: 9848 KiB

[startup+462.3 s]
/proc/loadavg: 2.05 2.12 2.09 3/200 15829
/proc/meminfo: memFree=31272256/32770624 swapFree=66934168/67108860
[pid=15785] ppid=15782 vsize=33300 memory=9856 CPUtime=462.3 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 8824 0 0 0 46230 0 0 0 20 0 1 0 412110305 34099200 2464 33554432000 4194304 6210612 140733409275952 140733409273912 5074258 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8325 2464 862 493 0 1630 0
Current children cumulated CPU time: 462.3 s
Current children cumulated vsize: 33300 KiB
Current children cumulated memory: 9856 KiB


################
# More data... #
################

/proc/15785/statm: 8325 2467 862 493 0 1630 0
Current children cumulated CPU time: 1722.34 s
Current children cumulated vsize: 33300 KiB
Current children cumulated memory: 9868 KiB

[startup+1782.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 15970
/proc/meminfo: memFree=31271284/32770624 swapFree=66934180/67108860
[pid=15785] ppid=15782 vsize=33300 memory=9868 CPUtime=1782.34 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 17770 0 0 0 178233 1 0 0 20 0 1 0 412110305 34099200 2467 33554432000 4194304 6210612 140733409275952 140733409273912 5076008 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8325 2467 862 493 0 1630 0
Current children cumulated CPU time: 1782.34 s
Current children cumulated vsize: 33300 KiB
Current children cumulated memory: 9868 KiB

[startup+1842.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 15974
/proc/meminfo: memFree=31271716/32770624 swapFree=66934180/67108860
[pid=15785] ppid=15782 vsize=33300 memory=9868 CPUtime=1842.34 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 18056 0 0 0 184233 1 0 0 20 0 1 0 412110305 34099200 2467 33554432000 4194304 6210612 140733409275952 140733409273912 5078905 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8325 2467 862 493 0 1630 0
Current children cumulated CPU time: 1842.34 s
Current children cumulated vsize: 33300 KiB
Current children cumulated memory: 9868 KiB

[startup+1902.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 15980
/proc/meminfo: memFree=31271220/32770624 swapFree=66934180/67108860
[pid=15785] ppid=15782 vsize=33300 memory=9868 CPUtime=1902.35 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 18343 0 0 0 190234 1 0 0 20 0 1 0 412110305 34099200 2467 33554432000 4194304 6210612 140733409275952 140733409273912 5078572 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8325 2467 862 493 0 1630 0
Current children cumulated CPU time: 1902.35 s
Current children cumulated vsize: 33300 KiB
Current children cumulated memory: 9868 KiB

[startup+1962.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 15984
/proc/meminfo: memFree=31271212/32770624 swapFree=66934180/67108860
[pid=15785] ppid=15782 vsize=33300 memory=9868 CPUtime=1962.35 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 18832 0 0 0 196234 1 0 0 20 0 1 0 412110305 34099200 2467 33554432000 4194304 6210612 140733409275952 140733409273912 5082119 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8325 2467 862 493 0 1630 0
Current children cumulated CPU time: 1962.35 s
Current children cumulated vsize: 33300 KiB
Current children cumulated memory: 9868 KiB

[startup+2022.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 15988
/proc/meminfo: memFree=31271756/32770624 swapFree=66934180/67108860
[pid=15785] ppid=15782 vsize=33300 memory=9868 CPUtime=2022.35 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 19117 0 0 0 202234 1 0 0 20 0 1 0 412110305 34099200 2467 33554432000 4194304 6210612 140733409275952 140733409273912 5082119 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8325 2467 862 493 0 1630 0
Current children cumulated CPU time: 2022.35 s
Current children cumulated vsize: 33300 KiB
Current children cumulated memory: 9868 KiB

[startup+2082.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 16002
/proc/meminfo: memFree=31271600/32770624 swapFree=66934180/67108860
[pid=15785] ppid=15782 vsize=33300 memory=9868 CPUtime=2082.35 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 19402 0 0 0 208234 1 0 0 20 0 1 0 412110305 34099200 2467 33554432000 4194304 6210612 140733409275952 140733409273912 5074840 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8325 2467 862 493 0 1630 0
Current children cumulated CPU time: 2082.35 s
Current children cumulated vsize: 33300 KiB
Current children cumulated memory: 9868 KiB

[startup+2142.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 16006
/proc/meminfo: memFree=31270784/32770624 swapFree=66934180/67108860
[pid=15785] ppid=15782 vsize=33300 memory=9868 CPUtime=2142.35 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 19891 0 0 0 214234 1 0 0 20 0 1 0 412110305 34099200 2467 33554432000 4194304 6210612 140733409275952 140733409273912 5076034 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8325 2467 862 493 0 1630 0
Current children cumulated CPU time: 2142.35 s
Current children cumulated vsize: 33300 KiB
Current children cumulated memory: 9868 KiB

[startup+2202.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 16010
/proc/meminfo: memFree=31271188/32770624 swapFree=66934180/67108860
[pid=15785] ppid=15782 vsize=33300 memory=9868 CPUtime=2202.35 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 20179 0 0 0 220234 1 0 0 20 0 1 0 412110305 34099200 2467 33554432000 4194304 6210612 140733409275952 140733409273912 5082388 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8325 2467 862 493 0 1630 0
Current children cumulated CPU time: 2202.35 s
Current children cumulated vsize: 33300 KiB
Current children cumulated memory: 9868 KiB

[startup+2262.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 16016
/proc/meminfo: memFree=31270800/32770624 swapFree=66934180/67108860
[pid=15785] ppid=15782 vsize=33300 memory=9868 CPUtime=2262.36 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 20466 0 0 0 226235 1 0 0 20 0 1 0 412110305 34099200 2467 33554432000 4194304 6210612 140733409275952 140733409273912 5076066 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8325 2467 862 493 0 1630 0
Current children cumulated CPU time: 2262.36 s
Current children cumulated vsize: 33300 KiB
Current children cumulated memory: 9868 KiB

[startup+2322.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 16020
/proc/meminfo: memFree=31270932/32770624 swapFree=66934180/67108860
[pid=15785] ppid=15782 vsize=33300 memory=9868 CPUtime=2322.36 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 20750 0 0 0 232235 1 0 0 20 0 1 0 412110305 34099200 2467 33554432000 4194304 6210612 140733409275952 140733409273912 5082450 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8325 2467 862 493 0 1630 0
Current children cumulated CPU time: 2322.36 s
Current children cumulated vsize: 33300 KiB
Current children cumulated memory: 9868 KiB

[startup+2382.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 16024
/proc/meminfo: memFree=31271644/32770624 swapFree=66934180/67108860
[pid=15785] ppid=15782 vsize=33300 memory=9868 CPUtime=2382.36 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 21241 0 0 0 238235 1 0 0 20 0 1 0 412110305 34099200 2467 33554432000 4194304 6210612 140733409275952 140733409273912 5185901 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8325 2467 862 493 0 1630 0
Current children cumulated CPU time: 2382.36 s
Current children cumulated vsize: 33300 KiB
Current children cumulated memory: 9868 KiB

[startup+2442.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 16030
/proc/meminfo: memFree=31271440/32770624 swapFree=66934180/67108860
[pid=15785] ppid=15782 vsize=33300 memory=9868 CPUtime=2442.36 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 21528 0 0 0 244235 1 0 0 20 0 1 0 412110305 34099200 2467 33554432000 4194304 6210612 140733409275952 140733409273912 5080992 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8325 2467 862 493 0 1630 0
Current children cumulated CPU time: 2442.36 s
Current children cumulated vsize: 33300 KiB
Current children cumulated memory: 9868 KiB

[startup+2502.3 s]
/proc/loadavg: 1.78 1.97 2.03 2/192 16080
/proc/meminfo: memFree=31282172/32770624 swapFree=66934180/67108860
[pid=15785] ppid=15782 vsize=33300 memory=9868 CPUtime=2502.36 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 21816 0 0 0 250235 1 0 0 20 0 1 0 412110305 34099200 2467 33554432000 4194304 6210612 140733409275952 140733409273912 5074256 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8325 2467 862 493 0 1630 0
Current children cumulated CPU time: 2502.36 s
Current children cumulated vsize: 33300 KiB
Current children cumulated memory: 9868 KiB



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+2520 s]*
/proc/loadavg: 1.61 1.92 2.01 2/193 16082
/proc/meminfo: memFree=31282872/32770624 swapFree=66934180/67108860
[pid=15785] ppid=15782 vsize=33300 memory=9868 CPUtime=2520.07 cores=0,2,4,6
/proc/15785/stat : 15785 (cosoco) R 15782 15785 14972 0 -1 4202496 22105 0 0 0 252006 1 0 0 20 0 1 0 412110305 34099200 2467 33554432000 4194304 6210612 140733409275952 140733409273912 5082455 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 9199616 140733409285131 140733409285233 140733409285233 140733409288142 0
/proc/15785/statm: 8325 2467 862 493 0 1630 0
Current children cumulated CPU time: 2520.07 s
Current children cumulated vsize: 33300 KiB
Current children cumulated memory: 9868 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

[startup+2520.01 s]
# the end of solver process 15785 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2520.07 s, system=0.014 s

Solver just ended.
??? end of watcher thread
Dumping a history of the last processes samples

Child status: 1
Real time (s): 2520.01
CPU time (s): 2520.08
CPU user time (s): 2520.07
CPU system time (s): 0.014
CPU usage (%): 100.003
Max. virtual memory (cumulated for all children) (KiB): 33300
Max. memory (cumulated for all children) (KiB): 9868

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2520.07
system time used= 0.014
maximum resident set size= 9936
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 22157
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 16
involuntary context switches= 639


# summary of solver processes directly reported to runsolver:
#   pid: 15785
#   total CPU time (s): 2520.08
#   total CPU user time (s): 2520.07
#   total CPU system time (s): 0.014

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 8.33513 second user time and 17.3029 second system time

The end

Launcher Data 

Begin job on node151 at 2019-09-13 09:05:49
IDJOB=4389570
IDBENCH=138939
IDSOLVER=2829
FILE ID=node151/4389570-1568358389
RUNJOBID= node151-1568353308-15003
PBS_JOBID= 21701185
Free space on /tmp= 56996 MiB

SOLVER NAME= cosoco 2
BENCH NAME= XCSP17/LatinSquare/LatinSquare-m1-gp/qwh-o30-h375-20.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4389570-1568358389/watcher-4389570-1568358389 -o /tmp/evaluation-result-4389570-1568358389/solver-4389570-1568358389 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node151-1568353308-15003 --watchdog 2580  cosoco -lc -model -restarts -stick HOME/instance-4389570-1568358389.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= cc39eb921a5cd1c2fcd1d541ff73edd4
RANDOM SEED=1758502985

node151.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.53
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5335.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.53
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5335.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.53
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5335.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.53
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5335.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        31280700 kB
MemAvailable:   31850852 kB
Buffers:          146796 kB
Cached:           714748 kB
SwapCached:        72052 kB
Active:           440144 kB
Inactive:         501196 kB
Active(anon):      15496 kB
Inactive(anon):    96488 kB
Active(file):     424648 kB
Inactive(file):   404708 kB
Unevictable:      111208 kB
Mlocked:          111208 kB
SwapTotal:      67108860 kB
SwapFree:       66934164 kB
Dirty:              2832 kB
Writeback:             0 kB
AnonPages:        148208 kB
Mapped:            37508 kB
Shmem:             23412 kB
Slab:             141368 kB
SReclaimable:     113264 kB
SUnreclaim:        28104 kB
KernelStack:        3776 kB
PageTables:         5716 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     534140 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     98304 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      138624 kB
DirectMap2M:    33406976 kB

Free space on /tmp at the end= 56996 MiB
End job on node151 at 2019-09-13 09:47:50