Trace number 3188565

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
QuteRSat 2011-03-01? (TO) 1300.07 1300.03

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v25000-c105000-S808762165-030-UNKNOWN.cnf
MD5SUMdd49a0090ce99bfddfd58615449391d9
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark25.1482
Satisfiable
(Un)Satisfiability was proved
Number of variables25000
Number of clauses105000
Sum of the clauses size315000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3105000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c Set FPU double precision.
0.10/0.14	c Parse Cnf success: #var: 25000 #clause: 105000
0.10/0.14	c Find Units :          0
0.10/0.14	c Find Invs  :          0
0.10/0.15	c Find Ands  :          0 AvgFaninNum: nan
0.10/0.15	c Find Xors  :          0 AvgFaninNum: nan
0.10/0.16	c Find Ite   :          0
0.19/0.21	c UnDirect variable: 0
0.19/0.21	c UnDirect to Direct Gate: 0
0.19/0.21	c Forced Direct Gate: 0 Unuse unDirect Gate: 0
0.19/0.21	c Break Cycle : 0
0.19/0.21	c Total Root : 0 Undef Var: 25000
0.19/0.21	c After covering, Total Root: 0 Leaf: 0
0.19/0.21	c Variable( cover/uncovered/unused ): 0 25000 0
0.19/0.21	c Unit Clauses: 0
0.19/0.21	c Binary Clauses( cover/uncovered ): 0 0
0.19/0.21	c Clauses( cover/uncovered ): 0 105000
0.19/0.21	c Total Clauses( cover/uncovered ): 0 105000 0%
0.19/0.21	c Covered Ands: 0 Invs: 0 Xors: 0 Ites: 0
0.29/0.31	c Uncover Clauses avg size: 3 max size: 3
0.29/0.31	c Final circuit: i/l/o/s   25000       0  105000    209991
0.29/0.31	c CnfToAig: Time: 0.31 s
0.39/0.48	c TmMap: Set choices finish.
0.49/0.56	c TmMap: Set big and: 105000
0.49/0.56	c TmMap: Origin: 234992 ClauseNum: 629973 --> New: 130001 ClauseNum: 420000
0.49/0.56	c TmMap: Time: 0.11
0.49/0.59	c Try to Add: #clause: 210000 #xorclause: 0
0.59/0.64	c #var: 25000 #clauses: 105001 Solving... 
1300.03/1300.00	c Time Out !!
1300.03/1300.00	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 893)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3188565-1302282661/watcher-3188565-1302282661 -o /tmp/evaluation-result-3188565-1302282661/solver-3188565-1302282661 -C 2400 -W 1300 -M 7680 qutersat_release HOME/instance-3188565-1302282661.cnf 1065490896 

running on 2 cores: 4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 7864320 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 7915520 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.01 4.02 4.16 3/181 14723
/proc/meminfo: memFree=15996220/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=33388 CPUtime=0 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 2090 0 0 0 0 0 0 0 25 0 1 0 339864141 34189312 2023 33554432000 4194304 6016207 140735087497744 18446744073709551615 228122713856 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 8347 2023 379 445 0 1891 0

[startup+0.107029 s]
/proc/loadavg: 4.01 4.02 4.16 3/181 14723
/proc/meminfo: memFree=15996220/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=60140 CPUtime=0.1 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 8662 0 0 0 9 1 0 0 25 0 1 0 339864141 61583360 8595 33554432000 4194304 6016207 140735087497744 18446744073709551615 4305741 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 15035 8595 379 445 0 8579 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 60140

[startup+0.20102 s]
/proc/loadavg: 4.01 4.02 4.16 3/181 14723
/proc/meminfo: memFree=15996220/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=60140 CPUtime=0.19 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 8924 0 0 0 18 1 0 0 25 0 1 0 339864141 61583360 8857 33554432000 4194304 6016207 140735087497744 18446744073709551615 4809356 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 15035 8857 396 445 0 8579 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 60140

[startup+0.301012 s]
/proc/loadavg: 4.01 4.02 4.16 3/181 14723
/proc/meminfo: memFree=15996220/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=66284 CPUtime=0.29 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 10464 0 0 0 28 1 0 0 25 0 1 0 339864141 67874816 10397 33554432000 4194304 6016207 140735087497744 18446744073709551615 4315424 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 16571 10397 400 445 0 10115 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 66284

[startup+0.700939 s]
/proc/loadavg: 4.01 4.02 4.16 3/181 14723
/proc/meminfo: memFree=15996220/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=108288 CPUtime=0.69 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 21253 0 0 0 67 2 0 0 25 0 1 0 339864141 110886912 21186 33554432000 4194304 6016207 140735087497744 18446744073709551615 4425167 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 27072 21186 691 445 0 20616 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 108288

[startup+1.5008 s]
/proc/loadavg: 4.01 4.02 4.16 5/190 14740
/proc/meminfo: memFree=15720068/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=108288 CPUtime=1.49 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 21282 0 0 0 147 2 0 0 25 0 1 0 339864141 110886912 21215 33554432000 4194304 6016207 140735087497744 18446744073709551615 4425824 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 27072 21215 719 445 0 20616 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 108288

[startup+3.10051 s]
/proc/loadavg: 4.01 4.02 4.16 5/190 14740
/proc/meminfo: memFree=15679396/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=141104 CPUtime=3.09 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 21741 0 0 0 307 2 0 0 25 0 1 0 339864141 144490496 21674 33554432000 4194304 6016207 140735087497744 18446744073709551615 4424229 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 35276 21674 719 445 0 28820 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 141104

[startup+6.30093 s]
/proc/loadavg: 4.01 4.02 4.16 5/190 14740
/proc/meminfo: memFree=15655972/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=141104 CPUtime=6.29 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 24297 0 0 0 627 2 0 0 25 0 1 0 339864141 144490496 24230 33554432000 4194304 6016207 140735087497744 18446744073709551615 4425840 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 35276 24230 719 445 0 28820 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 141104

[startup+12.7008 s]
/proc/loadavg: 4.01 4.02 4.16 5/190 14741
/proc/meminfo: memFree=15548008/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=228924 CPUtime=12.69 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 40175 0 0 0 1265 4 0 0 25 0 1 0 339864141 234418176 40108 33554432000 4194304 6016207 140735087497744 18446744073709551615 4424327 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 57231 40108 727 445 0 50775 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 228924

[startup+25.5005 s]
/proc/loadavg: 4.00 4.02 4.15 5/190 14741
/proc/meminfo: memFree=15377432/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=360992 CPUtime=25.49 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 47598 0 0 0 2544 5 0 0 25 0 1 0 339864141 369655808 47531 33554432000 4194304 6016207 140735087497744 18446744073709551615 4425149 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 90248 47531 727 445 0 83792 0
Current children cumulated CPU time (s) 25.49
Current children cumulated vsize (KiB) 360992

[startup+51.1009 s]
/proc/loadavg: 4.00 4.02 4.15 5/190 14742
/proc/meminfo: memFree=15223840/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=361248 CPUtime=51.1 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 65192 0 0 0 5101 9 0 0 25 0 1 0 339864141 369917952 65109 33554432000 4194304 6016207 140735087497744 18446744073709551615 4373657 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 90312 65109 727 445 0 83856 0
Current children cumulated CPU time (s) 51.1
Current children cumulated vsize (KiB) 361248

[startup+102.301 s]
/proc/loadavg: 4.08 4.03 4.14 5/190 14743
/proc/meminfo: memFree=14970368/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=389520 CPUtime=102.3 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 81883 0 0 0 10219 11 0 0 25 0 1 0 339864141 398868480 81587 33554432000 4194304 6016207 140735087497744 18446744073709551615 4378285 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 97380 81587 727 445 0 90924 0
Current children cumulated CPU time (s) 102.3
Current children cumulated vsize (KiB) 389520

[startup+162.301 s]
/proc/loadavg: 4.03 4.02 4.13 5/190 14744
/proc/meminfo: memFree=14845600/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=458416 CPUtime=162.29 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 88577 0 0 0 16217 12 0 0 25 0 1 0 339864141 469417984 88281 33554432000 4194304 6016207 140735087497744 18446744073709551615 4425132 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 114604 88281 727 445 0 108148 0
Current children cumulated CPU time (s) 162.29
Current children cumulated vsize (KiB) 458416

[startup+222.307 s]
/proc/loadavg: 4.05 4.03 4.13 5/190 14746
/proc/meminfo: memFree=14672572/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=569800 CPUtime=222.31 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 109292 0 0 0 22213 18 0 0 25 0 1 0 339864141 583475200 108769 33554432000 4194304 6016207 140735087497744 18446744073709551615 4424318 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 142450 108769 727 445 0 135994 0
Current children cumulated CPU time (s) 222.31
Current children cumulated vsize (KiB) 569800

[startup+282.301 s]
/proc/loadavg: 4.02 4.02 4.11 5/190 14748
/proc/meminfo: memFree=14543956/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=625344 CPUtime=282.3 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 110456 0 0 0 28212 18 0 0 25 0 1 0 339864141 640352256 109933 33554432000 4194304 6016207 140735087497744 18446744073709551615 4378514 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 156336 109933 727 445 0 149880 0
Current children cumulated CPU time (s) 282.3
Current children cumulated vsize (KiB) 625344

[startup+342.301 s]
/proc/loadavg: 4.05 4.03 4.11 5/190 14749
/proc/meminfo: memFree=14467032/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=625344 CPUtime=342.3 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 111426 0 0 0 34212 18 0 0 25 0 1 0 339864141 640352256 107458 33554432000 4194304 6016207 140735087497744 18446744073709551615 4425530 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 156336 107458 727 445 0 149880 0
Current children cumulated CPU time (s) 342.3
Current children cumulated vsize (KiB) 625344

[startup+402.301 s]
/proc/loadavg: 4.02 4.02 4.10 5/190 14751
/proc/meminfo: memFree=14407848/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=625344 CPUtime=402.3 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 113003 0 0 0 40212 18 0 0 25 0 1 0 339864141 640352256 109035 33554432000 4194304 6016207 140735087497744 18446744073709551615 4425027 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 156336 109035 727 445 0 149880 0
Current children cumulated CPU time (s) 402.3
Current children cumulated vsize (KiB) 625344

[startup+462.301 s]
/proc/loadavg: 4.00 4.02 4.09 5/190 14753
/proc/meminfo: memFree=14367888/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=625344 CPUtime=462.3 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 115598 0 0 0 46211 19 0 0 25 0 1 0 339864141 640352256 111630 33554432000 4194304 6016207 140735087497744 18446744073709551615 4385036 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 156336 111630 727 445 0 149880 0
Current children cumulated CPU time (s) 462.3
Current children cumulated vsize (KiB) 625344

[startup+522.3 s]
/proc/loadavg: 4.00 4.01 4.08 5/190 14754
/proc/meminfo: memFree=14281424/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=682800 CPUtime=522.3 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 133753 0 0 0 52205 25 0 0 25 0 1 0 339864141 699187200 129785 33554432000 4194304 6016207 140735087497744 18446744073709551615 4424323 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 170700 129785 727 445 0 164244 0
Current children cumulated CPU time (s) 522.3
Current children cumulated vsize (KiB) 682800

[startup+582.301 s]
/proc/loadavg: 4.00 4.01 4.08 5/190 14756
/proc/meminfo: memFree=14247044/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=682800 CPUtime=582.31 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 138078 0 0 0 58205 26 0 0 25 0 1 0 339864141 699187200 134110 33554432000 4194304 6016207 140735087497744 18446744073709551615 4425705 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 170700 134110 727 445 0 164244 0
Current children cumulated CPU time (s) 582.31
Current children cumulated vsize (KiB) 682800

[startup+642.301 s]
/proc/loadavg: 4.00 4.00 4.07 5/190 14757
/proc/meminfo: memFree=14232492/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=682800 CPUtime=642.31 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 139965 0 0 0 64205 26 0 0 25 0 1 0 339864141 699187200 135997 33554432000 4194304 6016207 140735087497744 18446744073709551615 4425389 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 170700 135997 727 445 0 164244 0
Current children cumulated CPU time (s) 642.31
Current children cumulated vsize (KiB) 682800

[startup+702.301 s]
/proc/loadavg: 4.04 4.01 4.07 5/190 14759
/proc/meminfo: memFree=14201208/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=682800 CPUtime=702.31 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 144152 0 0 0 70204 27 0 0 25 0 1 0 339864141 699187200 140184 33554432000 4194304 6016207 140735087497744 18446744073709551615 4425176 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 170700 140184 727 445 0 164244 0
Current children cumulated CPU time (s) 702.31
Current children cumulated vsize (KiB) 682800

[startup+762.301 s]
/proc/loadavg: 4.07 4.02 4.07 5/190 14761
/proc/meminfo: memFree=14188532/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=682800 CPUtime=762.32 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 144810 0 0 0 76205 27 0 0 25 0 1 0 339864141 699187200 140842 33554432000 4194304 6016207 140735087497744 18446744073709551615 4385057 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 170700 140842 727 445 0 164244 0
Current children cumulated CPU time (s) 762.32
Current children cumulated vsize (KiB) 682800

[startup+822.3 s]
/proc/loadavg: 4.02 4.02 4.06 5/190 14762
/proc/meminfo: memFree=14134056/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=682800 CPUtime=822.31 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 145751 0 0 0 82204 27 0 0 25 0 1 0 339864141 699187200 141783 33554432000 4194304 6016207 140735087497744 18446744073709551615 4425530 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 170700 141783 727 445 0 164244 0
Current children cumulated CPU time (s) 822.31
Current children cumulated vsize (KiB) 682800

[startup+882.307 s]
/proc/loadavg: 4.06 4.03 4.06 5/190 14764
/proc/meminfo: memFree=14099172/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=683184 CPUtime=882.32 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 153625 0 0 0 88204 28 0 0 25 0 1 0 339864141 699580416 149657 33554432000 4194304 6016207 140735087497744 18446744073709551615 4425254 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 170796 149657 727 445 0 164340 0
Current children cumulated CPU time (s) 882.32
Current children cumulated vsize (KiB) 683184

[startup+942.301 s]
/proc/loadavg: 4.02 4.02 4.06 5/190 14765
/proc/meminfo: memFree=14073220/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=683184 CPUtime=942.32 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 155710 0 0 0 94203 29 0 0 25 0 1 0 339864141 699580416 151742 33554432000 4194304 6016207 140735087497744 18446744073709551615 4425389 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 170796 151742 727 445 0 164340 0
Current children cumulated CPU time (s) 942.32
Current children cumulated vsize (KiB) 683184

[startup+1002.3 s]
/proc/loadavg: 4.01 4.02 4.05 5/190 14767
/proc/meminfo: memFree=14023216/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=683184 CPUtime=1002.31 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 159572 0 0 0 100202 29 0 0 25 0 1 0 339864141 699580416 155604 33554432000 4194304 6016207 140735087497744 18446744073709551615 4425856 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 170796 155604 727 445 0 164340 0
Current children cumulated CPU time (s) 1002.31
Current children cumulated vsize (KiB) 683184

[startup+1062.3 s]
/proc/loadavg: 4.00 4.01 4.04 5/190 14768
/proc/meminfo: memFree=14019584/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=683184 CPUtime=1062.32 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 159772 0 0 0 106203 29 0 0 25 0 1 0 339864141 699580416 155804 33554432000 4194304 6016207 140735087497744 18446744073709551615 4424223 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 170796 155804 727 445 0 164340 0
Current children cumulated CPU time (s) 1062.32
Current children cumulated vsize (KiB) 683184

[startup+1122.3 s]
/proc/loadavg: 4.00 4.00 4.04 5/190 14770
/proc/meminfo: memFree=13991772/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=926464 CPUtime=1122.32 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 166112 0 0 0 112202 30 0 0 25 0 1 0 339864141 948699136 162144 33554432000 4194304 6016207 140735087497744 18446744073709551615 4425709 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 231616 162144 727 445 0 225160 0
Current children cumulated CPU time (s) 1122.32
Current children cumulated vsize (KiB) 926464

[startup+1182.3 s]
/proc/loadavg: 4.00 4.00 4.03 5/190 14772
/proc/meminfo: memFree=13986904/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=1170252 CPUtime=1182.32 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 166571 0 0 0 118202 30 0 0 25 0 1 0 339864141 1198338048 162603 33554432000 4194304 6016207 140735087497744 18446744073709551615 228122412269 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 292563 162603 727 445 0 286107 0
Current children cumulated CPU time (s) 1182.32
Current children cumulated vsize (KiB) 1170252

[startup+1242.3 s]
/proc/loadavg: 4.00 4.00 4.03 5/190 14773
/proc/meminfo: memFree=13976700/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=1659800 CPUtime=1242.32 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 167849 0 0 0 124202 30 0 0 25 0 1 0 339864141 1699635200 163859 33554432000 4194304 6016207 140735087497744 18446744073709551615 4425530 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 414950 163859 727 445 0 408494 0
Current children cumulated CPU time (s) 1242.32
Current children cumulated vsize (KiB) 1659800



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1300 s]
/proc/loadavg: 4.00 4.00 4.02 4/188 14776
/proc/meminfo: memFree=14175324/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=2151824 CPUtime=1300.03 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 169865 0 0 0 129972 31 0 0 25 0 1 0 339864141 2203467776 165720 33554432000 4194304 6016207 140735087497744 18446744073709551615 4368949 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 537956 165720 727 445 0 531500 0
Current children cumulated CPU time (s) 1300.03
Current children cumulated vsize (KiB) 2151824

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1300 s]
/proc/loadavg: 4.00 4.00 4.02 4/188 14776
/proc/meminfo: memFree=14175324/32951124 swapFree=67111528/67111528
[pid=14723] ppid=14721 vsize=2151824 CPUtime=1300.03 cores=4,6
/proc/14723/stat : 14723 (qutersat_releas) R 14721 14723 14374 0 -1 4202496 169865 0 0 0 129972 31 0 0 25 0 1 0 339864141 2203467776 165720 33554432000 4194304 6016207 140735087497744 18446744073709551615 4368949 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14723/statm: 537956 165720 727 445 0 531500 0
Current children cumulated CPU time (s) 1300.03
Current children cumulated vsize (KiB) 2151824

Child status: 1
Real time (s): 1300.03
CPU time (s): 1300.07
CPU user time (s): 1299.73
CPU system time (s): 0.339948
CPU usage (%): 100.003
Max. virtual memory (cumulated for all children) (KiB): 2151824

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1299.73
system time used= 0.339948
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 169870
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 6
involuntary context switches= 1705

runsolver used 2.24766 second user time and 4.85126 second system time

The end

Launcher Data 

Begin job on node138 at 2011-04-08 19:11:02
IDJOB=3188565
IDBENCH=82696
IDSOLVER=1607
FILE ID=node138/3188565-1302282661
RUNJOBID= node138-1302280057-14393
PBS_JOBID= 12919056
Free space on /tmp= 70988 MiB

SOLVER NAME= QuteRSat 2011-03-01
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v25000-c105000-S808762165-030-UNKNOWN.cnf
COMMAND LINE= qutersat_release BENCHNAME RANDOMSEED
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3188565-1302282661/watcher-3188565-1302282661 -o /tmp/evaluation-result-3188565-1302282661/solver-3188565-1302282661 -C 2400 -W 1300 -M 7680  qutersat_release HOME/instance-3188565-1302282661.cnf 1065490896

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 7680 MiB
NBCORE= 2

MD5SUM BENCH= dd49a0090ce99bfddfd58615449391d9
RANDOM SEED=1065490896

node138.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      15996996 kB
Buffers:        507344 kB
Cached:       15961076 kB
SwapCached:          0 kB
Active:        9333492 kB
Inactive:      7232736 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      15996996 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:            6660 kB
Writeback:           0 kB
AnonPages:       98104 kB
Mapped:          16644 kB
Slab:           323580 kB
PageTables:       5148 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   244740 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 70980 MiB
End job on node138 at 2011-04-08 19:32:44