Trace number 3293706

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
CryptoMiniSat Strange-Night-1-mt? (TO) 4800.52 1201.8

General information on the benchmark

NameSAT-Race-2010-CNF/hardware-verification/
velev/11pipe_k.cnf
MD5SUM932387240a0c17f4c88af52c4ca151c6
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark325.87
Satisfiable
(Un)Satisfiability was proved
Number of variables89315
Number of clauses5584003
Sum of the clauses size16593705
Maximum clause length295
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 25473787
Number of clauses of size 337063
Number of clauses of size 4514
Number of clauses of size 5453
Number of clauses of size over 572185

Solver Data 

0.00/0.00	c Outputting solution to console
0.00/0.00	c Using 4 threads
0.00/0.00	c This is CryptoMiniSat 2.9.0
0.00/0.00	c compiled with gcc version 4.4.5
0.00/0.00	c WARNING: for repeatability, setting FPU to use double precision
0.00/0.00	c Reading file 'HOME/instance-3293706-1304181809.cnf'
0.00/0.00	c -- header says num vars:          89315
0.00/0.00	c -- header says num clauses:     5584003
3.81/1.40	c -- clauses added:            0 learnts,      5584003 normals,            0 xors
3.81/1.40	c -- vars added      89315
3.81/1.42	c Parsing time:  5.05 s
3.81/1.42	c  N st     0         0     89315    110215   5473787         0  16593704         0   no data   no data  --
9.88/2.51	c asymm  cl-useful: 13/13/110213 lits-rem:157 time: 4.33
35.82/9.06	c Flit:  1550 Blit:      0 bXBeca:    0 bXProp:   10 Bins:      0 BRemL:      0 BRemN:      0 P: 200.0M T: 25.82
36.22/9.14	c Cleaning up after failed var search:     0.32 s 
36.62/9.28	c Replacing       12 vars Replaced     1726 lits Time:     0.57 s 
40.19/10.12	c bin-w-bin subsume rem          695 bins  time:  1.80 s
46.20/11.63	c subs with bin:        0  lits-rem:        24  v-fix:    0  time:  6.01 s
51.77/13.01	c Subs w/ non-existent bins:      0 l-rem:     88 v-fix:     0 done:  85537 time:  5.51 s
52.97/13.30	c Removed useless bin:       0  fixed:     0  props:  20.00M  time:  1.19 s
680.52/170.44	c lits-rem:         0  cl-subs:    14342  v-elim:   7223  v-fix:    0  time: 626.86 s
680.92/170.50	c Finding binary XORs  T:     0.27 s  found:      32
681.32/170.65	c Replacing       32 vars Replaced      320 lits Time:     0.60 s 
682.92/171.01	c Finding non-binary XORs:     0.68 s (found:       0, avg size: nan)
682.92/171.01	c Calc default polars -  time:   0.28 s pos:    5345 undec:    8953 neg:   75017
682.92/171.01	c =========================================================================================
682.92/171.01	c types(t): F = full restart, N = normal restart
682.92/171.01	c types(t): S = simplification begin/end, E = solution found
682.92/171.01	c restart types(rt): st = static, dy = dynamic
682.92/171.01	c  t rt  Rest     Confl      Vars   NormCls    BinCls   Learnts    ClLits    LtLits
682.92/171.01	c  B st     0         0     80485     84758   5266837         0  14426441         0   no data   no data  --
690.91/173.06	c  N dy    12      4244     80485     84758   5266946      4110  14426441    577248     83.12     90.41  --
700.91/175.55	c  N dy    19     11202     80451     84758   5269555     10706  14426435   1709596     60.46     64.31  --
708.08/177.34	c  N dy    28     16675     80450     84758   5269797     15844  14426435   2326477     53.61     56.68  --
716.47/179.43	c  N dy    43     21027     80439     84758   5270334     10489  14426425    700970     52.84     57.02  --
722.88/181.10	c  F st    57     25000     80434     84758   5270814     14285  14426425   1551889     51.68     53.23  --
726.85/182.08	c  N st    58     25250     80434     84758   5264369     14531  14413471   1618311     51.68   no data  --
735.24/184.19	c  N dy    63     30003     80434     84758   5264481     19104  14413471   4604067     51.68   no data  --
735.24/184.19	c  S st    63     30003     80434     84758   5264481     19104  14413471   4604067     51.68   no data  --
739.23/185.12	c  S st    63     30503     80417     84758   5265051     19391  14413471   4620265     51.68   no data  --
739.63/185.24	c x-sub:     0 x-cut:      0 vfix:      0 v-elim:      0 locsubst:     0 time:   0.50
3146.24/787.68	c Flit:  2044 Blit:     29 bXBeca:    0 bXProp:    2 Bins:  44259 BRemL:      1 BRemN:      0 P: 200.0M T: 2405.62
3146.64/787.76	c Cleaning up after failed var search:     0.32 s 
3147.44/787.96	c Replacing        3 vars Replaced     1867 lits Time:     0.76 s 
3151.83/789.10	c bin-w-bin subsume rem          257 bins  time:  2.79 s
3157.45/790.41	c subs with bin:        0  lits-rem:        23  v-fix:    0  time:  5.26 s
3159.42/790.90	c Subs w/ non-existent bins:      0 l-rem:    241 v-fix:     0 done:   8709 time:  1.84 s
3160.62/791.24	c Removed useless bin:    1177  fixed:     0  props:  20.00M  time:  1.36 s
3212.56/804.22	c lits-rem:       309  cl-subs:     6746  v-elim:   2689  v-fix:    0  time: 51.80 s
3214.16/804.65	c asymm  cl-useful: 1333/1361/141610 lits-rem:72081 time: 1.71
3214.57/804.75	c  N dy    63     30503     75649    141606   4970917     12702  14697901   1536611     51.68   no data  --
3220.55/806.20	c  N dy    78     35198     75645    141606   4971079     17185  14697900   2055594     50.47     53.72  --
3224.95/807.33	c  N dy    84     39445     69926    141606   5020690     21268  14697897   2803771     48.62     52.07  --
3228.17/808.13	c Finding binary XORs  T:     0.18 s  found:      64
3229.75/808.59	c  N dy    93     43644     69879    140479   4282847     25312  13034765   3291332     47.14     50.38  --
3232.16/809.19	c  S st    99     45754     69879    140479   4282907     27328  13034764   3636009     46.51     34.33  --
3234.94/809.89	c  S st    99     46256     69818    140479   4283057     27660  13034761   3646169     46.51   no data  --
3235.34/809.98	c x-sub:     0 x-cut:      0 vfix:      0 v-elim:      0 locsubst:     0 time:   0.38
4131.10/1034.17	c Flit:  1220 Blit:      3 bXBeca:    0 bXProp:   10 Bins:   9285 BRemL:     35 BRemN:      0 P: 200.1M T: 894.57
4131.49/1034.26	c Cleaning up after failed var search:     0.37 s 
4132.30/1034.44	c Replacing       74 vars Replaced    29791 lits Time:     0.70 s 
4135.89/1035.34	c bin-w-bin subsume rem         7794 bins  time:  2.20 s
4148.28/1038.45	c subs with bin:      101  lits-rem:     11380  v-fix:    0  time: 12.43 s
4153.87/1039.88	c Subs w/ non-existent bins:    668 l-rem:   7308 v-fix:     0 done:  66135 time:  5.64 s
4155.49/1040.27	c Removed useless bin:       2  fixed:     0  props:  20.00M  time:  1.14 s
4213.80/1054.88	c lits-rem:      3209  cl-subs:     2024  v-elim:   1082  v-fix:    0  time: 58.36 s
4217.02/1055.63	c vivif2 --  cl tried    59095 cl shrink     2528 lits rem       6064 time: 2.99
4218.59/1056.02	c vivif2 --  cl tried    26200 cl shrink    13109 lits rem     184734 time: 1.56
4219.79/1056.35	c asymm  cl-useful: 272/356/168239 lits-rem:4056 time: 1.32
4221.00/1056.66	c calculated reachability. Time: 0.93
4221.00/1056.66	c  N dy    99     46256     67438    168238   3999393     26183  12647622   3143918     46.51   no data  --
4228.99/1058.62	c  N dy   112     50332     67438    168238   3999522     15644  12647618   1036605     45.41     50.36  --
4236.58/1060.55	c  N dy   125     54377     65104    163966   3725720     19470  11930211   1502100     44.65     47.01  --
4244.56/1062.51	c  N dy   145     58900     65104    163966   3725806     23783  11930206   1934173     44.05     48.45  --
4251.36/1064.25	c  N dy   160     63157     64936    163966   3726312     27805  11930204   2312930     43.20     45.80  --
4257.35/1065.71	c  N dy   171     67354     64928    163966   3726683     31856  11930200   2749829     42.62     45.00  --
4260.56/1066.50	c  S st   179     69384     64928    163966   3726740     33780  11930200   2956185     42.28     35.35  --
4263.35/1067.24	c  S st   179     69884     64924    163966   3726858     34153  11930186   2963217     42.28   no data  --
4263.75/1067.32	c x-sub:     0 x-cut:      0 vfix:      0 v-elim:      0 locsubst:     0 time:   0.35
4263.75/1067.33	c Cache  BProp: 2 Set: 0 BXProp: 0 T: 0.04
4748.37/1188.62	c Flit:  1969 Blit:     17 bXBeca:    0 bXProp:   66 Bins:   6857 BRemL:     44 BRemN:      0 P: 200.0M T: 483.25
4748.37/1188.70	c Cleaning up after failed var search:     0.32 s 
4749.15/1188.86	c Replacing       67 vars Replaced    47881 lits Time:     0.64 s 
4751.95/1189.58	c bin-w-bin subsume rem         5658 bins  time:  1.72 s
4757.56/1190.94	c subs with bin:      472  lits-rem:     52185  v-fix:    0  time:  5.42 s
4761.14/1191.86	c Subs w/ non-existent bins:    464 l-rem:   6756 v-fix:     0 done:  32560 time:  3.60 s
4762.74/1192.25	c Removed useless bin:    7781  fixed:     0  props:  20.00M  time:  1.45 s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3293706-1304181809/watcher-3293706-1304181809 -o /tmp/evaluation-result-3293706-1304181809/solver-3293706-1304181809 -C 4800 -W 1300 -M 15500 ./strangenight1-mt --threads=4 HOME/instance-3293706-1304181809.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.17 8.14 7.75 5/180 31507
/proc/meminfo: memFree=30023392/32951124 swapFree=67103364/67111528
[pid=31507] ppid=31505 vsize=51236 CPUtime=0 cores=0,2,4,6
/proc/31507/stat : 31507 (strangenight1-m) R 31505 31507 30707 0 -1 4202496 412 0 0 0 0 0 0 0 23 0 4 0 608411668 52465664 351 33554432000 4194304 4650419 140734302927808 18446744073709551615 47606436136876 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/31507/statm: 12809 351 303 112 0 7774 0
[pid=31507/tid=31508] ppid=31505 vsize=51236 CPUtime=0 cores=0,2,4,6
/proc/31507/task/31508/stat : 31508 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 0 0 0 0 0 0 0 0 24 0 4 0 608411669 52465664 351 33554432000 4194304 4650419 140734302927808 18446744073709551615 224284982257 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=31507/tid=31509] ppid=31505 vsize=51236 CPUtime=0 cores=0,2,4,6
/proc/31507/task/31509/stat : 31509 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 0 0 0 0 0 0 0 0 24 0 4 0 608411669 52465664 351 33554432000 4194304 4650419 140734302927808 18446744073709551615 224284982257 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=31507/tid=31510] ppid=31505 vsize=51236 CPUtime=0 cores=0,2,4,6
/proc/31507/task/31510/stat : 31510 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 0 0 0 0 0 0 0 0 24 0 4 0 608411669 52465664 351 33554432000 4194304 4650419 140734302927808 18446744073709551615 224284982257 0 0 4096 2 0 0 0 -1 2 0 0 0

[startup+0.0745711 s]
/proc/loadavg: 8.17 8.14 7.75 5/180 31507
/proc/meminfo: memFree=30023392/32951124 swapFree=67103364/67111528
[pid=31507] ppid=31505 vsize=290524 CPUtime=0.25 cores=0,2,4,6
/proc/31507/stat : 31507 (strangenight1-m) R 31505 31507 30707 0 -1 4202496 9764 0 0 0 22 3 0 0 18 0 4 0 608411668 297496576 9263 33554432000 4194304 4650419 140734302927808 18446744073709551615 224284969143 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/31507/statm: 72631 9385 343 112 0 67596 0
[pid=31507/tid=31508] ppid=31505 vsize=290656 CPUtime=0.06 cores=0,2,4,6
/proc/31507/task/31508/stat : 31508 (strangenight1-m) D 31505 31507 30707 0 -1 4202560 2432 0 0 0 6 0 0 0 18 0 4 0 608411669 297631744 9647 33554432000 4194304 4650419 140734302927808 18446744073709551615 224284622944 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31507/tid=31509] ppid=31505 vsize=290940 CPUtime=0.05 cores=0,2,4,6
/proc/31507/task/31509/stat : 31509 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 2454 0 0 0 5 0 0 0 18 0 4 0 608411669 297922560 9867 33554432000 4194304 4650419 140734302927808 18446744073709551615 224284969143 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=31507/tid=31510] ppid=31505 vsize=291096 CPUtime=0.06 cores=0,2,4,6
/proc/31507/task/31510/stat : 31510 (strangenight1-m) D 31505 31507 30707 0 -1 4202560 2608 0 0 0 6 0 0 0 18 0 4 0 608411669 298082304 9951 33554432000 4194304 4650419 140734302927808 18446744073709551615 224284622603 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 0.25
Current children cumulated vsize (KiB) 290524

[startup+0.13588 s]
/proc/loadavg: 8.17 8.14 7.75 5/180 31507
/proc/meminfo: memFree=30023392/32951124 swapFree=67103364/67111528
[pid=31507] ppid=31505 vsize=301660 CPUtime=0.52 cores=0,2,4,6
/proc/31507/stat : 31507 (strangenight1-m) D 31505 31507 30707 0 -1 4202496 18161 0 0 0 47 5 0 0 18 0 4 0 608411668 308899840 17120 33554432000 4194304 4650419 140734302927808 18446744073709551615 224284587041 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/31507/statm: 75415 17279 344 112 0 70380 0
[pid=31507/tid=31508] ppid=31505 vsize=301660 CPUtime=0.11 cores=0,2,4,6
/proc/31507/task/31508/stat : 31508 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 4466 0 0 0 11 0 0 0 18 0 4 0 608411669 308899840 17279 33554432000 4194304 4650419 140734302927808 18446744073709551615 224284983722 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31507/tid=31509] ppid=31505 vsize=301940 CPUtime=0.12 cores=0,2,4,6
/proc/31507/task/31509/stat : 31509 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 4428 0 0 0 11 1 0 0 18 0 4 0 608411669 309186560 17366 33554432000 4194304 4650419 140734302927808 18446744073709551615 224284969143 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=31507/tid=31510] ppid=31505 vsize=301940 CPUtime=0.12 cores=0,2,4,6
/proc/31507/task/31510/stat : 31510 (strangenight1-m) D 31505 31507 30707 0 -1 4202560 4645 0 0 0 12 0 0 0 18 0 4 0 608411669 309186560 17493 33554432000 4194304 4650419 140734302927808 18446744073709551615 224284586693 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 0.52
Current children cumulated vsize (KiB) 301660

[startup+0.320866 s]
/proc/loadavg: 8.17 8.14 7.75 5/180 31507
/proc/meminfo: memFree=30023392/32951124 swapFree=67103364/67111528
[pid=31507] ppid=31505 vsize=324560 CPUtime=1.22 cores=0,2,4,6
/proc/31507/stat : 31507 (strangenight1-m) D 31505 31507 30707 0 -1 4202496 39046 0 0 0 111 11 0 0 18 0 4 0 608411668 332349440 36904 33554432000 4194304 4650419 140734302927808 18446744073709551615 224284587041 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/31507/statm: 81178 37256 344 112 0 76143 0
[pid=31507/tid=31508] ppid=31505 vsize=324712 CPUtime=0.3 cores=0,2,4,6
/proc/31507/task/31508/stat : 31508 (strangenight1-m) D 31505 31507 30707 0 -1 4202560 9687 0 0 0 28 2 0 0 18 0 4 0 608411669 332505088 37283 33554432000 4194304 4650419 140734302927808 18446744073709551615 224284587041 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31507/tid=31509] ppid=31505 vsize=324900 CPUtime=0.3 cores=0,2,4,6
/proc/31507/task/31509/stat : 31509 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 9676 0 0 0 27 3 0 0 18 0 4 0 608411669 332697600 37305 33554432000 4194304 4650419 140734302927808 18446744073709551615 4545738 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=31507/tid=31510] ppid=31505 vsize=324900 CPUtime=0.3 cores=0,2,4,6
/proc/31507/task/31510/stat : 31510 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 10039 0 0 0 29 1 0 0 18 0 4 0 608411669 332697600 37344 33554432000 4194304 4650419 140734302927808 18446744073709551615 224284622951 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 1.22
Current children cumulated vsize (KiB) 324560

[startup+0.778032 s]
/proc/loadavg: 8.17 8.14 7.75 5/180 31507
/proc/meminfo: memFree=30023392/32951124 swapFree=67103364/67111528
[pid=31507] ppid=31505 vsize=709108 CPUtime=3.02 cores=0,2,4,6
/proc/31507/stat : 31507 (strangenight1-m) R 31505 31507 30707 0 -1 4202496 89716 0 0 0 277 25 0 0 18 0 4 0 608411668 726126592 85373 33554432000 4194304 4650419 140734302927808 18446744073709551615 224284969143 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/31507/statm: 177517 85491 344 112 0 172482 0
[pid=31507/tid=31508] ppid=31505 vsize=710068 CPUtime=0.75 cores=0,2,4,6
/proc/31507/task/31508/stat : 31508 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 22106 0 0 0 70 5 0 0 18 0 4 0 608411669 727109632 85642 33554432000 4194304 4650419 140734302927808 18446744073709551615 4373800 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31507/tid=31509] ppid=31505 vsize=710068 CPUtime=0.75 cores=0,2,4,6
/proc/31507/task/31509/stat : 31509 (strangenight1-m) D 31505 31507 30707 0 -1 4202560 22325 0 0 0 69 6 0 0 18 0 4 0 608411669 727109632 86397 33554432000 4194304 4650419 140734302927808 18446744073709551615 224284969143 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=31507/tid=31510] ppid=31505 vsize=710068 CPUtime=0.75 cores=0,2,4,6
/proc/31507/task/31510/stat : 31510 (strangenight1-m) D 31505 31507 30707 0 -1 4202560 23359 0 0 0 71 4 0 0 18 0 4 0 608411669 727109632 86397 33554432000 4194304 4650419 140734302927808 18446744073709551615 4520364 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3.02
Current children cumulated vsize (KiB) 709108

[startup+1.8342 s]
/proc/loadavg: 8.32 8.17 7.76 8/184 31511
/proc/meminfo: memFree=29527608/32951124 swapFree=67103364/67111528
[pid=31507] ppid=31505 vsize=995528 CPUtime=7.21 cores=0,2,4,6
/proc/31507/stat : 31507 (strangenight1-m) R 31505 31507 30707 0 -1 4202496 175863 0 0 0 671 50 0 0 19 0 4 0 608411668 1019420672 166833 33554432000 4194304 4650419 140734302927808 18446744073709551615 4370621 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31507/statm: 248882 166833 368 112 0 243847 0
[pid=31507/tid=31508] ppid=31505 vsize=995528 CPUtime=1.79 cores=0,2,4,6
/proc/31507/task/31508/stat : 31508 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 44303 0 0 0 168 11 0 0 18 0 4 0 608411669 1019420672 166833 33554432000 4194304 4650419 140734302927808 18446744073709551615 4370888 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31507/tid=31509] ppid=31505 vsize=995528 CPUtime=1.79 cores=0,2,4,6
/proc/31507/task/31509/stat : 31509 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 44277 0 0 0 167 12 0 0 18 0 4 0 608411669 1019420672 166833 33554432000 4194304 4650419 140734302927808 18446744073709551615 4370618 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=31507/tid=31510] ppid=31505 vsize=995528 CPUtime=1.8 cores=0,2,4,6
/proc/31507/task/31510/stat : 31510 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 42596 0 0 0 171 9 0 0 19 0 4 0 608411669 1019420672 166834 33554432000 4194304 4650419 140734302927808 18446744073709551615 4364883 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 7.21
Current children cumulated vsize (KiB) 995528

[startup+3.10099 s]
/proc/loadavg: 8.32 8.17 7.76 9/184 31511
/proc/meminfo: memFree=29315584/32951124 swapFree=67103364/67111528
[pid=31507] ppid=31505 vsize=984984 CPUtime=12.27 cores=0,2,4,6
/proc/31507/stat : 31507 (strangenight1-m) R 31505 31507 30707 0 -1 4202496 203221 0 0 0 1172 55 0 0 21 0 4 0 608411668 1008623616 176662 33554432000 4194304 4650419 140734302927808 18446744073709551615 4370749 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31507/statm: 246246 176662 372 112 0 241211 0
[pid=31507/tid=31508] ppid=31505 vsize=984984 CPUtime=3.05 cores=0,2,4,6
/proc/31507/task/31508/stat : 31508 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 53029 0 0 0 293 12 0 0 20 0 4 0 608411669 1008623616 176662 33554432000 4194304 4650419 140734302927808 18446744073709551615 4370329 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31507/tid=31509] ppid=31505 vsize=984984 CPUtime=3.06 cores=0,2,4,6
/proc/31507/task/31509/stat : 31509 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 52980 0 0 0 291 15 0 0 20 0 4 0 608411669 1008623616 176662 33554432000 4194304 4650419 140734302927808 18446744073709551615 4370621 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=31507/tid=31510] ppid=31505 vsize=984984 CPUtime=3.06 cores=0,2,4,6
/proc/31507/task/31510/stat : 31510 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 43776 0 0 0 297 9 0 0 21 0 4 0 608411669 1008623616 176662 33554432000 4194304 4650419 140734302927808 18446744073709551615 4365323 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 12.27
Current children cumulated vsize (KiB) 984984

[startup+6.30045 s]
/proc/loadavg: 8.37 8.18 7.77 9/184 31511
/proc/meminfo: memFree=29293988/32951124 swapFree=67103364/67111528
[pid=31507] ppid=31505 vsize=998244 CPUtime=25.04 cores=0,2,4,6
/proc/31507/stat : 31507 (strangenight1-m) R 31505 31507 30707 0 -1 4202496 207392 0 0 0 2447 57 0 0 25 0 4 0 608411668 1022201856 180833 33554432000 4194304 4650419 140734302927808 18446744073709551615 4370522 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31507/statm: 249561 180833 373 112 0 244526 0
[pid=31507/tid=31508] ppid=31505 vsize=998244 CPUtime=6.25 cores=0,2,4,6
/proc/31507/task/31508/stat : 31508 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 53279 0 0 0 613 12 0 0 25 0 4 0 608411669 1022201856 180833 33554432000 4194304 4650419 140734302927808 18446744073709551615 4370585 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31507/tid=31509] ppid=31505 vsize=998244 CPUtime=6.25 cores=0,2,4,6
/proc/31507/task/31509/stat : 31509 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 53228 0 0 0 610 15 0 0 25 0 4 0 608411669 1022201856 180833 33554432000 4194304 4650419 140734302927808 18446744073709551615 4370522 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=31507/tid=31510] ppid=31505 vsize=998244 CPUtime=6.27 cores=0,2,4,6
/proc/31507/task/31510/stat : 31510 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 47199 0 0 0 617 10 0 0 25 0 4 0 608411669 1022201856 180833 33554432000 4194304 4650419 140734302927808 18446744073709551615 4363880 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 25.04
Current children cumulated vsize (KiB) 998244

[startup+12.7004 s]
/proc/loadavg: 8.42 8.19 7.77 9/184 31511
/proc/meminfo: memFree=28945352/32951124 swapFree=67103364/67111528
[pid=31507] ppid=31505 vsize=1399408 CPUtime=50.59 cores=0,2,4,6
/proc/31507/stat : 31507 (strangenight1-m) R 31505 31507 30707 0 -1 4202496 293263 0 0 0 4984 75 0 0 25 0 4 0 608411668 1432993792 266704 33554432000 4194304 4650419 140734302927808 18446744073709551615 4448092 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31507/statm: 349852 266704 386 112 0 344817 0
[pid=31507/tid=31508] ppid=31505 vsize=1399408 CPUtime=12.64 cores=0,2,4,6
/proc/31507/task/31508/stat : 31508 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 81202 0 0 0 1247 17 0 0 25 0 4 0 608411669 1432993792 266704 33554432000 4194304 4650419 140734302927808 18446744073709551615 4447103 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31507/tid=31509] ppid=31505 vsize=1399408 CPUtime=12.65 cores=0,2,4,6
/proc/31507/task/31509/stat : 31509 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 80897 0 0 0 1243 22 0 0 25 0 4 0 608411669 1432993792 266704 33554432000 4194304 4650419 140734302927808 18446744073709551615 4241230 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=31507/tid=31510] ppid=31505 vsize=1399408 CPUtime=12.67 cores=0,2,4,6
/proc/31507/task/31510/stat : 31510 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 49834 0 0 0 1256 11 0 0 25 0 4 0 608411669 1432993792 266704 33554432000 4194304 4650419 140734302927808 18446744073709551615 4365535 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 50.59
Current children cumulated vsize (KiB) 1399408

[startup+25.5013 s]
/proc/loadavg: 8.36 8.19 7.77 9/184 31511
/proc/meminfo: memFree=28931580/32951124 swapFree=67103364/67111528
[pid=31507] ppid=31505 vsize=1406668 CPUtime=101.71 cores=0,2,4,6
/proc/31507/stat : 31507 (strangenight1-m) R 31505 31507 30707 0 -1 4202496 298023 0 0 0 10094 77 0 0 25 0 4 0 608411668 1440428032 271461 33554432000 4194304 4650419 140734302927808 18446744073709551615 4432392 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31507/statm: 351667 271461 391 112 0 346632 0
[pid=31507/tid=31508] ppid=31505 vsize=1406668 CPUtime=25.44 cores=0,2,4,6
/proc/31507/task/31508/stat : 31508 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 81607 0 0 0 2527 17 0 0 25 0 4 0 608411669 1440428032 271461 33554432000 4194304 4650419 140734302927808 18446744073709551615 4432392 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31507/tid=31509] ppid=31505 vsize=1406668 CPUtime=25.44 cores=0,2,4,6
/proc/31507/task/31509/stat : 31509 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 81305 0 0 0 2522 22 0 0 25 0 4 0 608411669 1440428032 271461 33554432000 4194304 4650419 140734302927808 18446744073709551615 4433081 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=31507/tid=31510] ppid=31505 vsize=1406668 CPUtime=25.47 cores=0,2,4,6
/proc/31507/task/31510/stat : 31510 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 53378 0 0 0 2534 13 0 0 25 0 4 0 608411669 1440428032 271461 33554432000 4194304 4650419 140734302927808 18446744073709551615 4364883 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 101.71
Current children cumulated vsize (KiB) 1406668

[startup+51.101 s]
/proc/loadavg: 8.24 8.17 7.78 9/184 31511
/proc/meminfo: memFree=28933312/32951124 swapFree=67103364/67111528
[pid=31507] ppid=31505 vsize=1420864 CPUtime=203.96 cores=0,2,4,6
/proc/31507/stat : 31507 (strangenight1-m) R 31505 31507 30707 0 -1 4202496 311705 0 0 0 20314 82 0 0 25 0 4 0 608411668 1454964736 273204 33554432000 4194304 4650419 140734302927808 18446744073709551615 4432376 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31507/statm: 355216 273204 391 112 0 350181 0
[pid=31507/tid=31508] ppid=31505 vsize=1420864 CPUtime=51.04 cores=0,2,4,6
/proc/31507/task/31508/stat : 31508 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 81607 0 0 0 5087 17 0 0 25 0 4 0 608411669 1454964736 273204 33554432000 4194304 4650419 140734302927808 18446744073709551615 4432392 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31507/tid=31509] ppid=31505 vsize=1420864 CPUtime=51.04 cores=0,2,4,6
/proc/31507/task/31509/stat : 31509 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 81305 0 0 0 5082 22 0 0 25 0 4 0 608411669 1454964736 273204 33554432000 4194304 4650419 140734302927808 18446744073709551615 4432392 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=31507/tid=31510] ppid=31505 vsize=1420864 CPUtime=51.07 cores=0,2,4,6
/proc/31507/task/31510/stat : 31510 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 67060 0 0 0 5089 18 0 0 25 0 4 0 608411669 1454964736 273204 33554432000 4194304 4650419 140734302927808 18446744073709551615 4365323 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 203.96
Current children cumulated vsize (KiB) 1420864

[startup+102.3 s]
/proc/loadavg: 8.14 8.15 7.79 9/184 31513
/proc/meminfo: memFree=28397284/32951124 swapFree=67103364/67111528
[pid=31507] ppid=31505 vsize=1919156 CPUtime=408.49 cores=0,2,4,6
/proc/31507/stat : 31507 (strangenight1-m) R 31505 31507 30707 0 -1 4202496 434529 0 0 0 40737 112 0 0 25 0 4 0 608411668 1965215744 394482 33554432000 4194304 4650419 140734302927808 18446744073709551615 4433081 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31507/statm: 479789 394482 392 112 0 474754 0
[pid=31507/tid=31508] ppid=31505 vsize=1919156 CPUtime=102.25 cores=0,2,4,6
/proc/31507/task/31508/stat : 31508 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 84058 0 0 0 10206 19 0 0 25 0 4 0 608411669 1965215744 394482 33554432000 4194304 4650419 140734302927808 18446744073709551615 4432392 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31507/tid=31509] ppid=31505 vsize=1919156 CPUtime=102.24 cores=0,2,4,6
/proc/31507/task/31509/stat : 31509 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 84097 0 0 0 10201 23 0 0 25 0 4 0 608411669 1965215744 394482 33554432000 4194304 4650419 140734302927808 18446744073709551615 4432392 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=31507/tid=31510] ppid=31505 vsize=1919156 CPUtime=102.27 cores=0,2,4,6
/proc/31507/task/31510/stat : 31510 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 181811 0 0 0 10183 44 0 0 25 0 4 0 608411669 1965215744 394482 33554432000 4194304 4650419 140734302927808 18446744073709551615 4433081 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 408.49
Current children cumulated vsize (KiB) 1919156

[startup+162.3 s]

################
# More data... #
################

Current children cumulated CPU time (s) 2565.74
Current children cumulated vsize (KiB) 2092760

[startup+702.301 s]
/proc/loadavg: 4.59 6.79 7.44 5/175 31547
/proc/meminfo: memFree=28352128/32951124 swapFree=67103364/67111528
[pid=31507] ppid=31505 vsize=2623640 CPUtime=2805.43 cores=0,2,4,6
/proc/31507/stat : 31507 (strangenight1-m) R 31505 31507 30707 0 -1 4202496 1196121 0 0 0 280277 266 0 0 25 0 4 0 608411668 2686607360 537317 33554432000 4194304 4650419 140734302927808 18446744073709551615 4230784 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31507/statm: 655910 537317 421 112 0 650875 0
[pid=31507/tid=31508] ppid=31505 vsize=2623640 CPUtime=702.25 cores=0,2,4,6
/proc/31507/task/31508/stat : 31508 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 303291 0 0 0 70139 86 0 0 25 0 4 0 608411669 2686607360 537317 33554432000 4194304 4650419 140734302927808 18446744073709551615 4432376 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31507/tid=31509] ppid=31505 vsize=2623640 CPUtime=702.23 cores=0,2,4,6
/proc/31507/task/31509/stat : 31509 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 318502 0 0 0 70159 64 0 0 25 0 4 0 608411669 2686607360 537317 33554432000 4194304 4650419 140734302927808 18446744073709551615 4366933 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=31507/tid=31510] ppid=31505 vsize=2623640 CPUtime=702.27 cores=0,2,4,6
/proc/31507/task/31510/stat : 31510 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 305917 0 0 0 70166 61 0 0 25 0 4 0 608411669 2686607360 537317 33554432000 4194304 4650419 140734302927808 18446744073709551615 4365321 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 2805.43
Current children cumulated vsize (KiB) 2623640

[startup+762.301 s]
/proc/loadavg: 4.22 6.28 7.22 5/175 31549
/proc/meminfo: memFree=28323712/32951124 swapFree=67103364/67111528
[pid=31507] ppid=31505 vsize=2640792 CPUtime=3045.15 cores=0,2,4,6
/proc/31507/stat : 31507 (strangenight1-m) R 31505 31507 30707 0 -1 4202496 1269095 0 0 0 304234 281 0 0 25 0 4 0 608411668 2704171008 543555 33554432000 4194304 4650419 140734302927808 18446744073709551615 4230546 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31507/statm: 660198 543555 421 112 0 655163 0
[pid=31507/tid=31508] ppid=31505 vsize=2640792 CPUtime=762.25 cores=0,2,4,6
/proc/31507/task/31508/stat : 31508 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 317209 0 0 0 76137 88 0 0 25 0 4 0 608411669 2704171008 543555 33554432000 4194304 4650419 140734302927808 18446744073709551615 224284580848 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31507/tid=31509] ppid=31505 vsize=2640792 CPUtime=762.23 cores=0,2,4,6
/proc/31507/task/31509/stat : 31509 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 345605 0 0 0 76154 69 0 0 25 0 4 0 608411669 2704171008 543555 33554432000 4194304 4650419 140734302927808 18446744073709551615 4231031 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=31507/tid=31510] ppid=31505 vsize=2640792 CPUtime=762.28 cores=0,2,4,6
/proc/31507/task/31510/stat : 31510 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 330595 0 0 0 76161 67 0 0 25 0 4 0 608411669 2704171008 543555 33554432000 4194304 4650419 140734302927808 18446744073709551615 4231007 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3045.15
Current children cumulated vsize (KiB) 2640792

[startup+822.301 s]
/proc/loadavg: 4.12 5.88 7.03 5/175 31551
/proc/meminfo: memFree=28191872/32951124 swapFree=67103364/67111528
[pid=31507] ppid=31505 vsize=2713264 CPUtime=3284.9 cores=0,2,4,6
/proc/31507/stat : 31507 (strangenight1-m) R 31505 31507 30707 0 -1 4202496 1386500 0 0 0 328168 322 0 0 25 0 4 0 608411668 2778382336 576395 33554432000 4194304 4650419 140734302927808 18446744073709551615 4230798 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31507/statm: 678316 576395 395 112 0 673281 0
[pid=31507/tid=31508] ppid=31505 vsize=2713264 CPUtime=822.25 cores=0,2,4,6
/proc/31507/task/31508/stat : 31508 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 317209 0 0 0 82137 88 0 0 25 0 4 0 608411669 2778382336 576395 33554432000 4194304 4650419 140734302927808 18446744073709551615 4230798 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31507/tid=31509] ppid=31505 vsize=2713264 CPUtime=822.23 cores=0,2,4,6
/proc/31507/task/31509/stat : 31509 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 378110 0 0 0 82149 74 0 0 25 0 4 0 608411669 2778382336 576395 33554432000 4194304 4650419 140734302927808 18446744073709551615 4236868 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=31507/tid=31510] ppid=31505 vsize=2713264 CPUtime=822.28 cores=0,2,4,6
/proc/31507/task/31510/stat : 31510 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 332372 0 0 0 82161 67 0 0 25 0 4 0 608411669 2778382336 576395 33554432000 4194304 4650419 140734302927808 18446744073709551615 4236868 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3284.9
Current children cumulated vsize (KiB) 2713264

[startup+882.301 s]
/proc/loadavg: 4.04 5.53 6.83 5/175 31553
/proc/meminfo: memFree=28158124/32951124 swapFree=67103364/67111528
[pid=31507] ppid=31505 vsize=2786892 CPUtime=3524.62 cores=0,2,4,6
/proc/31507/stat : 31507 (strangenight1-m) R 31505 31507 30707 0 -1 4202496 1457751 0 0 0 352130 332 0 0 25 0 4 0 608411668 2853777408 584452 33554432000 4194304 4650419 140734302927808 18446744073709551615 4236872 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31507/statm: 696723 584452 395 112 0 691688 0
[pid=31507/tid=31508] ppid=31505 vsize=2786892 CPUtime=882.25 cores=0,2,4,6
/proc/31507/task/31508/stat : 31508 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 317209 0 0 0 88137 88 0 0 25 0 4 0 608411669 2853777408 584452 33554432000 4194304 4650419 140734302927808 18446744073709551615 4230706 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31507/tid=31509] ppid=31505 vsize=2786892 CPUtime=882.23 cores=0,2,4,6
/proc/31507/task/31509/stat : 31509 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 411006 0 0 0 88145 78 0 0 25 0 4 0 608411669 2853777408 584452 33554432000 4194304 4650419 140734302927808 18446744073709551615 4227332 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=31507/tid=31510] ppid=31505 vsize=2786892 CPUtime=882.27 cores=0,2,4,6
/proc/31507/task/31510/stat : 31510 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 334348 0 0 0 88160 67 0 0 25 0 4 0 608411669 2853777408 584452 33554432000 4194304 4650419 140734302927808 18446744073709551615 4236891 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3524.62
Current children cumulated vsize (KiB) 2786892

[startup+942.307 s]
/proc/loadavg: 4.05 5.27 6.66 5/175 31553
/proc/meminfo: memFree=28125248/32951124 swapFree=67103364/67111528
[pid=31507] ppid=31505 vsize=2793828 CPUtime=3764.36 cores=0,2,4,6
/proc/31507/stat : 31507 (strangenight1-m) R 31505 31507 30707 0 -1 4202496 1528246 0 0 0 376096 340 0 0 25 0 4 0 608411668 2860879872 593035 33554432000 4194304 4650419 140734302927808 18446744073709551615 4230684 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31507/statm: 698457 593035 395 112 0 693422 0
[pid=31507/tid=31508] ppid=31505 vsize=2793828 CPUtime=942.25 cores=0,2,4,6
/proc/31507/task/31508/stat : 31508 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 317209 0 0 0 94137 88 0 0 25 0 4 0 608411669 2860879872 593035 33554432000 4194304 4650419 140734302927808 18446744073709551615 4230555 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31507/tid=31509] ppid=31505 vsize=2793828 CPUtime=942.24 cores=0,2,4,6
/proc/31507/task/31509/stat : 31509 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 443530 0 0 0 94142 82 0 0 25 0 4 0 608411669 2860879872 593035 33554432000 4194304 4650419 140734302927808 18446744073709551615 4230684 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=31507/tid=31510] ppid=31505 vsize=2793828 CPUtime=942.29 cores=0,2,4,6
/proc/31507/task/31510/stat : 31510 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 336161 0 0 0 94161 68 0 0 25 0 4 0 608411669 2860879872 593035 33554432000 4194304 4650419 140734302927808 18446744073709551615 4236872 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3764.36
Current children cumulated vsize (KiB) 2793828

[startup+1002.3 s]
/proc/loadavg: 4.10 5.05 6.49 5/175 31555
/proc/meminfo: memFree=27561148/32951124 swapFree=67103364/67111528
[pid=31507] ppid=31505 vsize=3577008 CPUtime=4004.05 cores=0,2,4,6
/proc/31507/stat : 31507 (strangenight1-m) R 31505 31507 30707 0 -1 4202496 1731135 0 0 0 400016 389 0 0 25 0 4 0 608411668 3662856192 729384 33554432000 4194304 4650419 140734302927808 18446744073709551615 4230546 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31507/statm: 894252 729384 395 112 0 889217 0
[pid=31507/tid=31508] ppid=31505 vsize=3577008 CPUtime=1002.25 cores=0,2,4,6
/proc/31507/task/31508/stat : 31508 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 331929 0 0 0 100124 101 0 0 25 0 4 0 608411669 3662856192 729384 33554432000 4194304 4650419 140734302927808 18446744073709551615 4365165 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31507/tid=31509] ppid=31505 vsize=3577008 CPUtime=1002.23 cores=0,2,4,6
/proc/31507/task/31509/stat : 31509 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 540696 0 0 0 100121 102 0 0 25 0 4 0 608411669 3662856192 729384 33554432000 4194304 4650419 140734302927808 18446744073709551615 4314702 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=31507/tid=31510] ppid=31505 vsize=3577008 CPUtime=1002.28 cores=0,2,4,6
/proc/31507/task/31510/stat : 31510 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 391153 0 0 0 100152 76 0 0 25 0 4 0 608411669 3662856192 729384 33554432000 4194304 4650419 140734302927808 18446744073709551615 4363756 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4004.05
Current children cumulated vsize (KiB) 3577008

[startup+1062.3 s]
/proc/loadavg: 4.11 4.87 6.34 5/175 31566
/proc/meminfo: memFree=27239212/32951124 swapFree=67103364/67111528
[pid=31507] ppid=31505 vsize=3975024 CPUtime=4243.76 cores=0,2,4,6
/proc/31507/stat : 31507 (strangenight1-m) R 31505 31507 30707 0 -1 4202496 1882458 0 0 0 423925 451 0 0 25 0 4 0 608411668 4070424576 813958 33554432000 4194304 4650419 140734302927808 18446744073709551615 4365321 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31507/statm: 993756 813958 395 112 0 988721 0
[pid=31507/tid=31508] ppid=31505 vsize=3975024 CPUtime=1062.25 cores=0,2,4,6
/proc/31507/task/31508/stat : 31508 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 339951 0 0 0 106122 103 0 0 25 0 4 0 608411669 4070424576 813958 33554432000 4194304 4650419 140734302927808 18446744073709551615 4236891 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31507/tid=31509] ppid=31505 vsize=3975024 CPUtime=1062.23 cores=0,2,4,6
/proc/31507/task/31509/stat : 31509 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 552156 0 0 0 106117 106 0 0 25 0 4 0 608411669 4070424576 813958 33554432000 4194304 4650419 140734302927808 18446744073709551615 4230556 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=31507/tid=31510] ppid=31505 vsize=3975024 CPUtime=1062.28 cores=0,2,4,6
/proc/31507/task/31510/stat : 31510 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 411118 0 0 0 106147 81 0 0 25 0 4 0 608411669 4070424576 813958 33554432000 4194304 4650419 140734302927808 18446744073709551615 4363339 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4243.76
Current children cumulated vsize (KiB) 3975024

[startup+1122.3 s]
/proc/loadavg: 4.10 4.73 6.20 5/175 31568
/proc/meminfo: memFree=27225808/32951124 swapFree=67103364/67111528
[pid=31507] ppid=31505 vsize=3981848 CPUtime=4483.49 cores=0,2,4,6
/proc/31507/stat : 31507 (strangenight1-m) R 31505 31507 30707 0 -1 4202496 1911405 0 0 0 447892 457 0 0 25 0 4 0 608411668 4077412352 817393 33554432000 4194304 4650419 140734302927808 18446744073709551615 4230556 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31507/statm: 995462 817393 395 112 0 990427 0
[pid=31507/tid=31508] ppid=31505 vsize=3981848 CPUtime=1122.25 cores=0,2,4,6
/proc/31507/task/31508/stat : 31508 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 339951 0 0 0 112122 103 0 0 25 0 4 0 608411669 4077412352 817393 33554432000 4194304 4650419 140734302927808 18446744073709551615 4236868 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31507/tid=31509] ppid=31505 vsize=3981848 CPUtime=1122.23 cores=0,2,4,6
/proc/31507/task/31509/stat : 31509 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 559941 0 0 0 112115 108 0 0 25 0 4 0 608411669 4077412352 817393 33554432000 4194304 4650419 140734302927808 18446744073709551615 4230684 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=31507/tid=31510] ppid=31505 vsize=3981848 CPUtime=1122.27 cores=0,2,4,6
/proc/31507/task/31510/stat : 31510 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 431204 0 0 0 112142 85 0 0 25 0 4 0 608411669 4077412352 817393 33554432000 4194304 4650419 140734302927808 18446744073709551615 4328947 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4483.49
Current children cumulated vsize (KiB) 3981848

[startup+1182.3 s]
/proc/loadavg: 4.10 4.61 6.06 5/175 31570
/proc/meminfo: memFree=26714288/32951124 swapFree=67103364/67111528
[pid=31507] ppid=31505 vsize=4649248 CPUtime=4723.19 cores=0,2,4,6
/proc/31507/stat : 31507 (strangenight1-m) R 31505 31507 30707 0 -1 4202496 2324521 0 0 0 471764 555 0 0 18 0 4 0 608411668 4760829952 946168 33554432000 4194304 4650419 140734302927808 18446744073709551615 4370730 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31507/statm: 1162312 946168 395 112 0 1157277 0
[pid=31507/tid=31508] ppid=31505 vsize=4649248 CPUtime=1182.25 cores=0,2,4,6
/proc/31507/task/31508/stat : 31508 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 414904 0 0 0 118081 144 0 0 25 0 4 0 608411669 4760829952 946168 33554432000 4194304 4650419 140734302927808 18446744073709551615 4365227 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31507/tid=31509] ppid=31505 vsize=4649248 CPUtime=1182.23 cores=0,2,4,6
/proc/31507/task/31509/stat : 31509 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 847251 0 0 0 118077 146 0 0 18 0 4 0 608411669 4760829952 946168 33554432000 4194304 4650419 140734302927808 18446744073709551615 4540553 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=31507/tid=31510] ppid=31505 vsize=4649248 CPUtime=1182.27 cores=0,2,4,6
/proc/31507/task/31510/stat : 31510 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 442174 0 0 0 118141 86 0 0 25 0 4 0 608411669 4760829952 946168 33554432000 4194304 4650419 140734302927808 18446744073709551615 4230548 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4723.19
Current children cumulated vsize (KiB) 4649248



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.6 s]
/proc/loadavg: 4.07 4.57 6.02 5/175 31570
/proc/meminfo: memFree=26499760/32951124 swapFree=67103364/67111528
[pid=31507] ppid=31505 vsize=4909780 CPUtime=4800.31 cores=0,2,4,6
/proc/31507/stat : 31507 (strangenight1-m) R 31505 31507 30707 0 -1 4202496 2395775 0 0 0 479436 595 0 0 25 0 4 0 608411668 5027614720 998385 33554432000 4194304 4650419 140734302927808 18446744073709551615 4428338 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31507/statm: 1227445 998385 395 112 0 1222410 0
[pid=31507/tid=31508] ppid=31505 vsize=4909780 CPUtime=1201.55 cores=0,2,4,6
/proc/31507/task/31508/stat : 31508 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 421999 0 0 0 120009 146 0 0 25 0 4 0 608411669 5027614720 998385 33554432000 4194304 4650419 140734302927808 18446744073709551615 4230557 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31507/tid=31509] ppid=31505 vsize=4909780 CPUtime=1201.53 cores=0,2,4,6
/proc/31507/task/31509/stat : 31509 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 858478 0 0 0 120004 149 0 0 25 0 4 0 608411669 5027614720 998385 33554432000 4194304 4650419 140734302927808 18446744073709551615 4520364 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=31507/tid=31510] ppid=31505 vsize=4909780 CPUtime=1201.59 cores=0,2,4,6
/proc/31507/task/31510/stat : 31510 (strangenight1-m) R 31505 31507 30707 0 -1 4202560 443131 0 0 0 120072 87 0 0 25 0 4 0 608411669 5027614720 998385 33554432000 4194304 4650419 140734302927808 18446744073709551615 4236872 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.31
Current children cumulated vsize (KiB) 4909780

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.71 s]
/proc/loadavg: 4.07 4.57 6.02 5/175 31570
/proc/meminfo: memFree=26499760/32951124 swapFree=67103364/67111528
[pid=31507] ppid=31505 vsize=0 CPUtime=4800.44 cores=0,2,4,6
/proc/31507/stat : 31507 (strangenight1-m) R 31505 31507 30707 0 -1 4203524 2395776 0 0 0 479439 605 0 0 25 0 1 0 608411668 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/31507/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 4800.44
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.8
CPU time (s): 4800.52
CPU user time (s): 4794.39
CPU system time (s): 6.13507
CPU usage (%): 399.446
Max. virtual memory (cumulated for all children) (KiB): 5010016

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4794.39
system time used= 6.13507
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2395776
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 18365
involuntary context switches= 19543

runsolver used 1.80872 second user time and 4.16437 second system time

The end

Launcher Data 

Begin job on node118 at 2011-04-30 18:43:30
IDJOB=3293706
IDBENCH=88167
IDSOLVER=1588
FILE ID=node118/3293706-1304181809
RUNJOBID= node118-1304180203-31288
PBS_JOBID= 13172422
Free space on /tmp= 71824 MiB

SOLVER NAME= CryptoMiniSat Strange-Night-1-mt
BENCH NAME= SAT-Race-2010-CNF/hardware-verification/velev/11pipe_k.cnf
COMMAND LINE= ./strangenight1-mt --threads=NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3293706-1304181809/watcher-3293706-1304181809 -o /tmp/evaluation-result-3293706-1304181809/solver-3293706-1304181809 -C 4800 -W 1300 -M 15500  ./strangenight1-mt --threads=4 HOME/instance-3293706-1304181809.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 932387240a0c17f4c88af52c4ca151c6
RANDOM SEED=1866599457

node118.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5374.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30023796 kB
Buffers:        443636 kB
Cached:        1602284 kB
SwapCached:       2708 kB
Active:         741728 kB
Inactive:      1826348 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30023796 kB
SwapTotal:    67111528 kB
SwapFree:     67103364 kB
Dirty:          109440 kB
Writeback:           0 kB
AnonPages:      520636 kB
Mapped:          15500 kB
Slab:           294612 kB
PageTables:       6064 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   806532 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71828 MiB
End job on node118 at 2011-04-30 19:03:34