Trace number 4408189

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
cosoco 2.0? (TO) 2519.81 2520.01

General information on the benchmark

NameLatinSquare/LatinSquare-m1-gs/
qwh-o100-h10000.xml
MD5SUM2d32aa9e5f9738deac3ebf02ed7c37b3
Bench CategoryCSP (decision problem)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark67.7052
Satisfiable
(Un)Satisfiability was proved
Number of variables10000
Number of constraints1
Number of domains1
Minimum domain size100
Maximum domain size100
Distribution of domain sizes[{"size":100,"count":10000}]
Minimum variable degree1
Maximum variable degree1
Distribution of variable degrees[{"degree":1,"count":10000}]
Minimum constraint arity10000
Maximum constraint arity10000
Distribution of constraint arities[{"arity":10000,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"allDifferent","count":1}]
Optimization problemNO
Type of objective

Solver Data 

0.21/0.27	c root propagations     : 0           
0.21/0.27	c restarts enabled, print stats at every restarts
0.21/0.27	c
0.21/0.27	c
0.21/0.27	c
0.21/0.27	c This is cosoco 2.00 --  
0.21/0.27	c
0.21/0.27	c command line: cosoco -lc -model -restarts -stick HOME/instance-4408189-1569811323.xml 
0.21/0.27	c ========================================[ Problem Statistics ]===========================================
0.21/0.27	c |                                                                                                       
0.21/0.27	c |  Parse time        :         0.05 s 
0.21/0.27	c |
0.21/0.27	c |               Variables: 10000
0.21/0.27	c |            Domain Sizes: 100..100
0.21/0.27	c |
0.21/0.27	c |             Constraints: 200
0.21/0.27	c |                   Arity: 100..100
0.21/0.27	c | 
0.21/0.27	c |                   Types: 
0.21/0.27	c |                          All Different: 200
0.21/0.27	c |                          
0.21/0.27	c =========================================================================================================
0.21/0.27	c conflicts      decisions      filterCalls    propagations   Root Prop.     Nogoods        min/max/avg depth
1.79/1.81	c 101            7057           91560          400629         0              0              2607   6875   6757
3.10/3.17	c 211            12399          171849         718296         0              0              993   5104   4799
4.88/4.94	c 332            18864          278034         1117539        0              0              5   6233   5626
6.79/6.89	c 465            25140          395833         1530565        0              0              6   6042   4747
8.09/8.19	c 611            28861          478482         1780509        0              0              7   3444   3242
10.09/10.16	c 771            34670          602138         2187301        0              0              12   5474   4796
11.88/11.96	c 947            39851          717349         2536429        0              0              19   4877   3673
13.38/13.43	c 1140           43901          814160         2813435        0              0              23   3732   3310
15.10/15.15	c 1352           48806          926885         3089449        0              0              29   4523   3235
16.68/16.73	c 1585           53344          1030960        3355814        0              0              30   4147   3287
17.99/18.07	c 1841           57136          1120085        3565359        0              0              32   3356   2864
19.78/19.84	c 2122           62229          1236104        3882516        0              0              40   4600   3475
21.68/21.70	c 2431           67534          1357587        4221698        0              0              49   4755   3669
23.39/23.47	c 2770           72751          1472612        4536761        0              0              63   4565   3657
25.38/25.42	c 3142           78716          1598055        4915387        0              0              90   5369   3916
27.18/27.25	c 3551           84272          1714448        5239450        0              0              42   4824   3912
29.18/29.23	c 4000           90234          1839863        5606956        0              0              33   5227   3747
31.09/31.14	c 4493           96063          1960332        5978811        0              0              22   5028   3992
32.78/32.85	c 5035           101153         2069728        6300438        0              0              28   4171   3617
34.88/34.96	c 5631           107965         2203246        6671626        0              0              32   5839   4147
36.57/36.68	c 6286           113521         2311221        6980818        0              0              48   4374   3747
38.88/38.97	c 7006           120655         2454343        7494754        0              0              32   5968   4647
40.88/40.91	c 7798           127244         2576367        7838315        0              0              37   5098   4160
42.58/42.68	c 8669           133049         2688629        8188784        0              0              36   4176   3747
45.58/45.68	c 9627           140469         2830442        8790694        0              0              24   5578   4662
46.99/47.10	c 10680          147623         2962801        9201916        0              0              29   5184   4423
49.28/49.35	c 11838          155013         3100736        9715879        0              0              35   5307   4186
51.58/51.62	c 13111          162396         3238109        10202451       0              0              36   5044   4112
54.08/54.14	c 14511          170869         3391863        10849079       0              0              31   5901   4828
55.98/56.04	c 16051          177570         3509366        11298840       0              0              27   3849   3519
58.27/58.33	c 17745          186390         3653274        11848957       0              0              26   5581   4546
60.78/60.87	c 19608          196729         3810961        12446497       0              0              32   6030   5122
63.48/63.52	c 21657          205931         3975017        13216231       0              0              42   5511   4943
66.27/66.30	c 23910          216624         4146935        14005865       0              0              58   6343   4788
68.77/68.88	c 26388          227218         4318916        15002315       0              0              9   6029   5176
71.57/71.60	c 29113          238357         4499153        15870701       0              0              10   6096   4629
74.27/74.33	c 32110          249848         4679692        16788632       0              0              4   5711   5235
77.26/77.35	c 35406          262368         4878059        17753599       0              0              4   5713   5083
80.56/80.65	c 39031          275823         5090006        19084508       0              0              5   6522   5047
84.26/84.38	c 43018          289791         5321792        20581938       0              0              7   6038   5526
87.76/87.83	c 47403          304889         5541896        21893797       0              0              6   6096   5050
91.16/91.20	c 52226          319374         5755070        23395783       0              0              20   5030   4757
94.76/94.81	c 57531          333912         5983572        24777105       0              0              30   4744   4522
100.26/100.38	c 63366          349735         6328515        27877340       0              0              21   4956   4709
104.46/104.55	c 69784          368178         6590055        29657834       0              0              20   6038   4871
109.35/109.46	c 76843          386122         6894472        31974434       0              0              24   4962   4746
114.56/114.63	c 84607          406958         7213864        35348136       0              0              34   5397   5187
120.85/120.94	c 93147          430510         7599923        38785466       0              0              19   6768   6237
126.65/126.77	c 102541         453657         7952670        41810397       0              0              12   6351   6099
135.26/135.34	c 112874         483001         8468941        46836980       0              0              13   6752   6398
142.95/143.04	c 124240         512876         8931444        51020193       0              0              27   6655   6292
149.84/149.92	c 136742         543492         9357913        54881243       0              0              34   5729   5551
160.55/160.68	c 150494         581832         9998051        59776403       0              0              13   6813   6377
170.55/170.63	c 165621         618661         10596617       65101823       0              0              16   5248   5140
182.75/182.84	c 182260         658285         11333677       72373571       0              0              17   5972   5766
193.14/193.23	c 200562         701358         11956857       77343194       0              0              19   4962   4875
204.54/204.60	c 220694         749653         12625533       83629101       0              0              23   5720   5608
217.83/217.98	c 242839         799222         13406058       91773643       0              0              16   6104   5926
234.43/234.54	c 267198         855972         14360636       100914981      0              0              21   7307   6572
246.63/246.70	c 293992         917115         15101110       106686195      0              0              28   5166   5084
261.83/261.99	c 323465         977337         16021626       116535738      0              0              20   6117   5843
280.32/280.48	c 355885         1049403        17089191       129313448      0              0              24   6167   6075
295.52/295.66	c 391547         1122353        17982025       138867923      0              0              12   5240   5182
312.72/312.86	c 430775         1202966        19020381       149509069      0              0              20   5279   5178
329.52/329.62	c 473925         1300313        20047358       156909728      0              0              21   3959   3932
352.62/352.76	c 521390         1402566        21355907       173541674      0              0              33   5970   5910
386.03/386.16	c 573601         1512050        23266404       196321660      0              0              30   6093   6021
415.71/415.87	c 631033         1623026        24962099       215905377      0              0              41   6285   6108
447.11/447.27	c 694208         1752875        26776454       237510098      0              0              42   5779   5714
489.61/489.72	c 763700         1901669        29170669       264559263      0              0              14   7088   5972
533.81/533.93	c 840141         2064357        31617711       299695528      0              0              22   7142   6694
580.21/580.35	c 924226         2247217        34336761       326323733      0              0              26   6635   6586
629.20/629.31	c 1016719        2444577        37151062       358142671      0              0              12   6773   6574
672.40/672.55	c 1118461        2656327        39734182       388563247      0              0              13   5786   5755
742.91/743.09	c 1230377        2900052        43609105       436172325      0              0              18   7711   7071
843.49/843.65	c 1353484        3213370        48950989       519431422      0              0              19   7234   7153
913.59/913.73	c 1488901        3467737        52910044       561979750      0              0              10   6342   6304
986.89/987.08	c 1637859        3776424        57189366       605371687      0              0              15   6405   6364
1066.78/1066.96	c 1801712        4124931        61665924       662881992      0              0              18   6645   6411
1158.77/1158.99	c 1981950        4599765        66916740       723631875      0              0              34   7267   7201
1310.26/1310.48	c 2180211        5015174        75276761       844503555      0              0              38   6937   6907
1425.76/1425.94	c 2398298        5497542        81722685       928733104      0              0              14   6489   5919
1535.15/1535.34	c 2638193        6020620        87961645       993579350      0              0              15   5801   5782
1676.64/1676.85	c 2902077        6588761        95917749       1084315098     0              0              17   6161   6144
1880.73/1880.93	c 3192349        7175030        107316766      1202462298     0              0              28   6401   6352
2022.32/2022.55	c 3511648        7768831        115518382      1291302069     0              0              14   6102   6088
2228.82/2229.08	c 3862876        8596414        127039355      1424196300     0              0              19   7146   -5699
2406.00/2406.27	c 4249226        9369127        137312432      1589773885     0              0              17   5968   -5161
2519.80/2520.00	
2519.80/2520.00	c *** INTERRUPTED ***
2519.80/2520.00	c restarts              : 88          
2519.80/2520.00	c conflicts             : 4455378      (1768 /sec)
2519.80/2520.00	c decisions             : 9810678      (3894 /sec)
2519.80/2520.00	c propagations          : 1659662616  (658738 /sec)
2519.80/2520.00	c root propagations     : 0           
2519.80/2520.00	c filter calls          : 143961878      (57140 /sec)
2519.80/2520.00	c useless filter calls  : 54320116       (37 %)
2519.80/2520.00	c Memory used           : 419.79 MB
2519.80/2520.00	c CPU time              : 2519.46 s
2519.80/2520.00	c real time             : 2520 s
2519.80/2520.00	
2519.80/2520.00	s UNKNOWN

Verifier Data


Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (15148 MiB free)
  memory of node 1: 16384 MiB (15588 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4408189-1569811323/watcher-4408189-1569811323 -o /tmp/evaluation-result-4408189-1569811323/solver-4408189-1569811323 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node104-1569783168-25107 --watchdog 2580 cosoco -lc -model -restarts -stick HOME/instance-4408189-1569811323.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=30386, runsolver pid=30383

[startup+0.100152 s]*
/proc/loadavg: 2.00 2.01 2.05 3/201 30387
/proc/meminfo: memFree=31451504/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=50848 memory=27388 CPUtime=0.1 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 7013 0 0 0 9 1 0 0 20 0 1 0 557388583 52068352 6847 33554432000 4194304 6211132 140720802074336 140720802070280 5397258 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 12712 6847 849 493 0 6017 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 50848 KiB
Current children cumulated memory: 27388 KiB

[startup+0.209868 s]*
/proc/loadavg: 2.00 2.01 2.05 3/201 30387
/proc/meminfo: memFree=31451504/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=50848 memory=27388 CPUtime=0.21 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 7013 0 0 0 20 1 0 0 20 0 1 0 557388583 52068352 6847 33554432000 4194304 6211132 140720802074336 140720802072600 5397258 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 12712 6847 849 493 0 6017 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 50848 KiB
Current children cumulated memory: 27388 KiB

[startup+0.309238 s]*
/proc/loadavg: 2.00 2.01 2.05 3/201 30387
/proc/meminfo: memFree=31451504/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=50848 memory=27388 CPUtime=0.31 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 7021 0 0 0 30 1 0 0 20 0 1 0 557388583 52068352 6847 33554432000 4194304 6211132 140720802074336 140720802072600 5078740 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 12712 6847 849 493 0 6017 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 50848 KiB
Current children cumulated memory: 27388 KiB

[startup+0.700378 s]
/proc/loadavg: 2.00 2.01 2.05 3/201 30387
/proc/meminfo: memFree=31450388/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=56832 memory=33060 CPUtime=0.69 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 8483 0 0 0 68 1 0 0 20 0 1 0 557388583 58195968 8265 33554432000 4194304 6211132 140720802074336 140720802071624 5078776 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 14208 8265 857 493 0 7513 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 56832 KiB
Current children cumulated memory: 33060 KiB

[startup+1.50038 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 30387
/proc/meminfo: memFree=31425492/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=144660 memory=116260 CPUtime=1.49 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 29246 0 0 0 145 4 0 0 20 0 1 0 557388583 148131840 29065 33554432000 4194304 6211132 140720802074336 140720802071624 5078765 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 36165 29065 857 493 0 29470 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 144660 KiB
Current children cumulated memory: 116260 KiB

[startup+3.10981 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 30387
/proc/meminfo: memFree=31287888/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=230000 memory=194548 CPUtime=3.1 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 65707 0 0 0 303 7 0 0 20 0 1 0 557388583 235520000 48637 33554432000 4194304 6211132 140720802074336 140720802071928 5409489 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 57500 48637 859 493 0 50805 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 230000 KiB
Current children cumulated memory: 194548 KiB

[startup+6.30036 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 30387
/proc/meminfo: memFree=31237668/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=279852 memory=242396 CPUtime=6.29 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 104416 0 0 0 619 10 0 0 20 0 1 0 557388583 286568448 60599 33554432000 4194304 6211132 140720802074336 140720802071624 5078762 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 69963 60599 859 493 0 63268 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 279852 KiB
Current children cumulated memory: 242396 KiB

[startup+12.7004 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 30387
/proc/meminfo: memFree=31188320/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=330464 memory=289816 CPUtime=12.68 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 153753 0 0 0 1254 14 0 0 20 0 1 0 557388583 338395136 72454 33554432000 4194304 6211132 140720802074336 140720802072296 5082487 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 82616 72454 859 493 0 75921 0
Current children cumulated CPU time: 12.68 s
Current children cumulated vsize: 330464 KiB
Current children cumulated memory: 289816 KiB

[startup+25.5003 s]
/proc/loadavg: 2.24 2.06 2.06 2/201 30449
/proc/meminfo: memFree=26869768/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=340220 memory=299200 CPUtime=25.48 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 196646 0 0 0 2531 17 0 0 20 0 1 0 557388583 348385280 74800 33554432000 4194304 6211132 140720802074336 140720802072296 5193728 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 85055 74800 859 493 0 78360 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 340220 KiB
Current children cumulated memory: 299200 KiB

[startup+51.1096 s]
/proc/loadavg: 2.99 2.25 2.12 2/201 30452
/proc/meminfo: memFree=25890104/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=355516 memory=314568 CPUtime=51.09 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 256963 0 0 0 5088 21 0 0 20 0 1 0 557388583 364048384 78642 33554432000 4194304 6211132 140720802074336 140720802072296 5193728 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 88879 78642 859 493 0 82184 0
Current children cumulated CPU time: 51.09 s
Current children cumulated vsize: 355516 KiB
Current children cumulated memory: 314568 KiB

[startup+102.3 s]
/proc/loadavg: 3.50 2.51 2.22 2/200 30520
/proc/meminfo: memFree=25001128/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=376692 memory=335356 CPUtime=102.26 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 324968 0 0 0 10200 26 0 0 20 0 1 0 557388583 385732608 83839 33554432000 4194304 6211132 140720802074336 140720802072296 5081220 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 94173 83839 859 493 0 87478 0
Current children cumulated CPU time: 102.26 s
Current children cumulated vsize: 376692 KiB
Current children cumulated memory: 335356 KiB

[startup+162.3 s]
/proc/loadavg: 3.42 2.72 2.31 3/200 30631
/proc/meminfo: memFree=30940920/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=391252 memory=355692 CPUtime=162.25 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 369518 0 0 0 16196 29 0 0 20 0 1 0 557388583 400642048 88923 33554432000 4194304 6211132 140720802074336 140720802072296 5079440 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 97813 88923 859 493 0 91118 0
Current children cumulated CPU time: 162.25 s
Current children cumulated vsize: 391252 KiB
Current children cumulated memory: 355692 KiB

[startup+222.3 s]
/proc/loadavg: 2.52 2.59 2.29 3/200 30635
/proc/meminfo: memFree=30311824/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=391532 memory=359588 CPUtime=222.23 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 386178 0 0 0 22193 30 0 0 20 0 1 0 557388583 400928768 89897 33554432000 4194304 6211132 140720802074336 140720802071624 5076077 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 97883 89897 859 493 0 91188 0
Current children cumulated CPU time: 222.23 s
Current children cumulated vsize: 391532 KiB
Current children cumulated memory: 359588 KiB

[startup+282.3 s]
/proc/loadavg: 2.19 2.48 2.28 3/200 30641
/proc/meminfo: memFree=29670860/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=402108 memory=376608 CPUtime=282.22 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 394336 0 0 0 28191 31 0 0 20 0 1 0 557388583 411758592 94152 33554432000 4194304 6211132 140720802074336 140720802072296 5081337 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 100527 94152 859 493 0 93832 0
Current children cumulated CPU time: 282.22 s
Current children cumulated vsize: 402108 KiB
Current children cumulated memory: 376608 KiB

[startup+342.3 s]
/proc/loadavg: 2.07 2.39 2.26 3/200 30645
/proc/meminfo: memFree=29042024/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=402108 memory=378236 CPUtime=342.22 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 395585 0 0 0 34191 31 0 0 20 0 1 0 557388583 411758592 94559 33554432000 4194304 6211132 140720802074336 140720802072296 5409485 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 100527 94559 859 493 0 93832 0
Current children cumulated CPU time: 342.22 s
Current children cumulated vsize: 402108 KiB
Current children cumulated memory: 378236 KiB

[startup+402.3 s]
/proc/loadavg: 2.03 2.32 2.24 3/200 30649
/proc/meminfo: memFree=28413392/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=402668 memory=378592 CPUtime=402.21 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 396818 0 0 0 40190 31 0 0 20 0 1 0 557388583 412332032 94648 33554432000 4194304 6211132 140720802074336 140720802071624 5079440 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 100667 94648 859 493 0 93972 0
Current children cumulated CPU time: 402.21 s
Current children cumulated vsize: 402668 KiB
Current children cumulated memory: 378592 KiB

[startup+462.3 s]
/proc/loadavg: 2.01 2.26 2.22 3/199 30655
/proc/meminfo: memFree=27783460/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=406648 memory=381928 CPUtime=462.21 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 398970 0 0 0 46190 31 0 0 20 0 1 0 557388583 416407552 95482 33554432000 4194304 6211132 140720802074336 140720802071624 5076405 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 101662 95482 859 493 0 94967 0
Current children cumulated CPU time: 462.21 s
Current children cumulated vsize: 406648 KiB
Current children cumulated memory: 381928 KiB


################
# More data... #
################

/proc/30386/statm: 107193 100035 859 493 0 100498 0
Current children cumulated CPU time: 1722.14 s
Current children cumulated vsize: 428772 KiB
Current children cumulated memory: 400140 KiB

[startup+1782.3 s]
/proc/loadavg: 2.09 2.17 2.15 3/200 30996
/proc/meminfo: memFree=31188320/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=428772 memory=400140 CPUtime=1782.13 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 433418 0 0 0 178179 34 0 0 20 0 1 0 557388583 439062528 100035 33554432000 4194304 6211132 140720802074336 140720802071624 5081706 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 107193 100035 859 493 0 100498 0
Current children cumulated CPU time: 1782.13 s
Current children cumulated vsize: 428772 KiB
Current children cumulated memory: 400140 KiB

[startup+1842.3 s]
/proc/loadavg: 2.09 2.15 2.15 3/200 31000
/proc/meminfo: memFree=31189352/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=428772 memory=400144 CPUtime=1842.13 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 433575 0 0 0 184179 34 0 0 20 0 1 0 557388583 439062528 100036 33554432000 4194304 6211132 140720802074336 140720802071624 5081762 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 107193 100036 859 493 0 100498 0
Current children cumulated CPU time: 1842.13 s
Current children cumulated vsize: 428772 KiB
Current children cumulated memory: 400144 KiB

[startup+1902.3 s]
/proc/loadavg: 2.03 2.12 2.14 3/200 31006
/proc/meminfo: memFree=31189652/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=428772 memory=400144 CPUtime=1902.13 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 435397 0 0 0 190178 35 0 0 20 0 1 0 557388583 439062528 100036 33554432000 4194304 6211132 140720802074336 140720802072296 5082423 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 107193 100036 859 493 0 100498 0
Current children cumulated CPU time: 1902.13 s
Current children cumulated vsize: 428772 KiB
Current children cumulated memory: 400144 KiB

[startup+1962.3 s]
/proc/loadavg: 2.01 2.10 2.13 3/200 31010
/proc/meminfo: memFree=31190432/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=428772 memory=400144 CPUtime=1962.13 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 435486 0 0 0 196178 35 0 0 20 0 1 0 557388583 439062528 100036 33554432000 4194304 6211132 140720802074336 140720802072296 5409485 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 107193 100036 859 493 0 100498 0
Current children cumulated CPU time: 1962.13 s
Current children cumulated vsize: 428772 KiB
Current children cumulated memory: 400144 KiB

[startup+2022.3 s]
/proc/loadavg: 2.00 2.08 2.12 3/199 31014
/proc/meminfo: memFree=31185036/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=428772 memory=405124 CPUtime=2022.12 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 435568 0 0 0 202177 35 0 0 20 0 1 0 557388583 439062528 101281 33554432000 4194304 6211132 140720802074336 140720802072296 5079248 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 107193 101281 859 493 0 100498 0
Current children cumulated CPU time: 2022.12 s
Current children cumulated vsize: 428772 KiB
Current children cumulated memory: 405124 KiB

[startup+2082.3 s]
/proc/loadavg: 2.00 2.07 2.12 3/198 31019
/proc/meminfo: memFree=31185344/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=428916 memory=405176 CPUtime=2082.12 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 436279 0 0 0 208177 35 0 0 20 0 1 0 557388583 439209984 101294 33554432000 4194304 6211132 140720802074336 140720802071624 5193728 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 107229 101294 859 493 0 100534 0
Current children cumulated CPU time: 2082.12 s
Current children cumulated vsize: 428916 KiB
Current children cumulated memory: 405176 KiB

[startup+2142.3 s]
/proc/loadavg: 2.00 2.06 2.11 3/198 31023
/proc/meminfo: memFree=31185204/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=428916 memory=405212 CPUtime=2142.12 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 436331 0 0 0 214177 35 0 0 20 0 1 0 557388583 439209984 101303 33554432000 4194304 6211132 140720802074336 140720802071624 5409485 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 107229 101303 859 493 0 100534 0
Current children cumulated CPU time: 2142.12 s
Current children cumulated vsize: 428916 KiB
Current children cumulated memory: 405212 KiB

[startup+2202.3 s]
/proc/loadavg: 2.00 2.04 2.11 3/198 31027
/proc/meminfo: memFree=31185384/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=428916 memory=405212 CPUtime=2202.11 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 436382 0 0 0 220176 35 0 0 20 0 1 0 557388583 439209984 101303 33554432000 4194304 6211132 140720802074336 140720802071624 5409485 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 107229 101303 859 493 0 100534 0
Current children cumulated CPU time: 2202.11 s
Current children cumulated vsize: 428916 KiB
Current children cumulated memory: 405212 KiB

[startup+2262.3 s]
/proc/loadavg: 2.00 2.04 2.10 3/198 31033
/proc/meminfo: memFree=31184324/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=429856 memory=406012 CPUtime=2262.11 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 437270 0 0 0 226176 35 0 0 20 0 1 0 557388583 440172544 101503 33554432000 4194304 6211132 140720802074336 140720802072296 5079360 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 107464 101503 859 493 0 100769 0
Current children cumulated CPU time: 2262.11 s
Current children cumulated vsize: 429856 KiB
Current children cumulated memory: 406012 KiB

[startup+2322.3 s]
/proc/loadavg: 2.00 2.03 2.09 3/199 31045
/proc/meminfo: memFree=31184312/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=429856 memory=406012 CPUtime=2322.11 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 437327 0 0 0 232176 35 0 0 20 0 1 0 557388583 440172544 101503 33554432000 4194304 6211132 140720802074336 140720802072296 5380299 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 107464 101503 859 493 0 100769 0
Current children cumulated CPU time: 2322.11 s
Current children cumulated vsize: 429856 KiB
Current children cumulated memory: 406012 KiB

[startup+2382.3 s]
/proc/loadavg: 2.00 2.03 2.09 3/199 31049
/proc/meminfo: memFree=31184248/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=429856 memory=406012 CPUtime=2382.1 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 437361 0 0 0 238175 35 0 0 20 0 1 0 557388583 440172544 101503 33554432000 4194304 6211132 140720802074336 140720802072296 5074593 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 107464 101503 859 493 0 100769 0
Current children cumulated CPU time: 2382.1 s
Current children cumulated vsize: 429856 KiB
Current children cumulated memory: 406012 KiB

[startup+2442.3 s]
/proc/loadavg: 2.00 2.02 2.08 3/199 31055
/proc/meminfo: memFree=31184348/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=429856 memory=406012 CPUtime=2442.1 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 438002 0 0 0 244175 35 0 0 20 0 1 0 557388583 440172544 101503 33554432000 4194304 6211132 140720802074336 140720802072296 5082427 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 107464 101503 859 493 0 100769 0
Current children cumulated CPU time: 2442.1 s
Current children cumulated vsize: 429856 KiB
Current children cumulated memory: 406012 KiB

[startup+2502.3 s]
/proc/loadavg: 2.00 2.02 2.08 3/199 31059
/proc/meminfo: memFree=31184684/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=429856 memory=406012 CPUtime=2502.09 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 438035 0 0 0 250174 35 0 0 20 0 1 0 557388583 440172544 101503 33554432000 4194304 6211132 140720802074336 140720802072296 5082759 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 107464 101503 859 493 0 100769 0
Current children cumulated CPU time: 2502.09 s
Current children cumulated vsize: 429856 KiB
Current children cumulated memory: 406012 KiB



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+2520 s]*
/proc/loadavg: 2.00 2.01 2.07 3/200 31062
/proc/meminfo: memFree=31185092/32770624 swapFree=66986684/67108860
[pid=30386] ppid=30383 vsize=429856 memory=406012 CPUtime=2519.8 cores=1,3,5,7
/proc/30386/stat : 30386 (cosoco) R 30383 30386 25075 0 -1 4202496 438061 0 0 0 251945 35 0 0 20 0 1 0 557388583 440172544 101503 33554432000 4194304 6211132 140720802074336 140720802072296 5081706 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 34504704 140720802079755 140720802079857 140720802079857 140720802082766 0
/proc/30386/statm: 107464 101503 859 493 0 100769 0
Current children cumulated CPU time: 2519.8 s
Current children cumulated vsize: 429856 KiB
Current children cumulated memory: 406012 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

[startup+2520.01 s]
# the end of solver process 30386 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2519.46 s, system=0.355976 s

Solver just ended.
??? end of watcher thread
Dumping a history of the last processes samples

Child status: 1
Real time (s): 2520.01
CPU time (s): 2519.81
CPU user time (s): 2519.46
CPU system time (s): 0.355976
CPU usage (%): 99.9921
Max. virtual memory (cumulated for all children) (KiB): 429856
Max. memory (cumulated for all children) (KiB): 406012

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2519.46
system time used= 0.355976
maximum resident set size= 406076
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 438112
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 67
involuntary context switches= 922


# summary of solver processes directly reported to runsolver:
#   pid: 30386
#   total CPU time (s): 2519.81
#   total CPU user time (s): 2519.46
#   total CPU system time (s): 0.355976

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 9.46503 second user time and 16.0523 second system time

The end

Launcher Data 

Begin job on node104 at 2019-09-30 03:31:36
IDJOB=4408189
IDBENCH=139122
IDSOLVER=2874
FILE ID=node104/4408189-1569811323
RUNJOBID= node104-1569783168-25107
PBS_JOBID= 21705839
Free space on /tmp= 62028 MiB

SOLVER NAME= cosoco 2.0
BENCH NAME= XCSP17/LatinSquare/LatinSquare-m1-gs/qwh-o100-h10000.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4408189-1569811323/watcher-4408189-1569811323 -o /tmp/evaluation-result-4408189-1569811323/solver-4408189-1569811323 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node104-1569783168-25107 --watchdog 2580  cosoco -lc -model -restarts -stick HOME/instance-4408189-1569811323.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 2d32aa9e5f9738deac3ebf02ed7c37b3
RANDOM SEED=1081948196

node104.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        31475228 kB
MemAvailable:   31706200 kB
Buffers:          199192 kB
Cached:           361792 kB
SwapCached:        22760 kB
Active:           433016 kB
Inactive:         289140 kB
Active(anon):     152196 kB
Inactive(anon):    91516 kB
Active(file):     280820 kB
Inactive(file):   197624 kB
Unevictable:      109484 kB
Mlocked:          109484 kB
SwapTotal:      67108860 kB
SwapFree:       66986684 kB
Dirty:              2760 kB
Writeback:             0 kB
AnonPages:        249760 kB
Mapped:            28520 kB
Shmem:             73664 kB
Slab:             166660 kB
SReclaimable:     136728 kB
SUnreclaim:        29932 kB
KernelStack:        3584 kB
PageTables:         5804 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     695168 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:    200704 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      124288 kB
DirectMap2M:    33421312 kB

Free space on /tmp at the end= 62024 MiB
End job on node104 at 2019-09-30 04:13:36