Trace number 3371847

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Lingeling 587f (fixed)? (TO) 1300 1300.02

General information on the benchmark

Nameindustrial/palacios/hard/
sortnet-8-ipc5-h19-sat.cnf
MD5SUM409f4401e9e8b60ee665ac5567a13e41
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark1110.09
Satisfiable
(Un)Satisfiability was proved
Number of variables359341
Number of clauses1254773
Sum of the clauses size3046189
Maximum clause length57
Minimum clause length1
Number of clauses of size 1256
Number of clauses of size 2896240
Number of clauses of size 3346435
Number of clauses of size 4830
Number of clauses of size 5571
Number of clauses of size over 510441

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3371847-1305430210/watcher-3371847-1305430210 -o /tmp/evaluation-result-3371847-1305430210/solver-3371847-1305430210 -C 2400 -W 1300 -M 7680 lingeling HOME/instance-3371847-1305430210.cnf 

running on 2 cores: 0,2

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 7864320 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 7915520 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.96 2.88 2.33 5/370 23475
/proc/meminfo: memFree=25018020/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=11796 CPUtime=0 cores=0,2
/proc/23475/stat : 23475 (runsolver) R 23473 23475 23384 0 -1 4202560 41 0 0 0 0 0 0 0 23 0 1 0 733251767 12079104 49 33554432000 4194304 5432360 140736098756496 18446744073709551615 4881623 0 2147483391 4096 24578 0 0 0 17 2 0 0 0
/proc/23475/statm: 2949 49 30 303 0 2643 0

[startup+0.104667 s]
/proc/loadavg: 3.96 2.88 2.33 5/370 23475
/proc/meminfo: memFree=25018020/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=15632 CPUtime=0.08 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 2592 0 0 0 8 0 0 0 23 0 1 0 733251767 16007168 2433 33554432000 4194304 4409284 140736408988928 18446744073709551615 251367546624 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 3908 2433 113 53 0 2331 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 15632

[startup+0.200658 s]
/proc/loadavg: 3.96 2.88 2.33 5/370 23475
/proc/meminfo: memFree=25018020/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=24592 CPUtime=0.17 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 4832 0 0 0 17 0 0 0 24 0 1 0 733251767 25182208 4673 33554432000 4194304 4409284 140736408988928 18446744073709551615 4197800 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 6148 4673 113 53 0 4571 0
Current children cumulated CPU time (s) 0.17
Current children cumulated vsize (KiB) 24592

[startup+0.300626 s]
/proc/loadavg: 3.96 2.88 2.33 5/370 23475
/proc/meminfo: memFree=25018020/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=42772 CPUtime=0.27 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 9377 0 0 0 26 1 0 0 25 0 1 0 733251767 43798528 9218 33554432000 4194304 4409284 140736408988928 18446744073709551615 251367546624 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 10693 9218 113 53 0 9116 0
Current children cumulated CPU time (s) 0.27
Current children cumulated vsize (KiB) 42772

[startup+0.700553 s]
/proc/loadavg: 3.96 2.88 2.33 5/370 23475
/proc/meminfo: memFree=25018020/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=68500 CPUtime=0.68 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 18150 0 0 0 65 3 0 0 25 0 1 0 733251767 70144000 15655 33554432000 4194304 4409284 140736408988928 18446744073709551615 4295821 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 17125 15655 118 53 0 15548 0
Current children cumulated CPU time (s) 0.68
Current children cumulated vsize (KiB) 68500

[startup+1.50041 s]
/proc/loadavg: 3.96 2.88 2.33 5/371 23476
/proc/meminfo: memFree=24923260/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=88124 CPUtime=1.47 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 28036 0 0 0 143 4 0 0 25 0 1 0 733251767 90238976 20539 33554432000 4194304 4409284 140736408988928 18446744073709551615 4284560 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 22031 20539 128 53 0 20454 0
Current children cumulated CPU time (s) 1.47
Current children cumulated vsize (KiB) 88124

[startup+3.1011 s]
/proc/loadavg: 3.96 2.88 2.33 6/371 23476
/proc/meminfo: memFree=24909748/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=88124 CPUtime=3.08 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 28040 0 0 0 303 5 0 0 25 0 1 0 733251767 90238976 20543 33554432000 4194304 4409284 140736408988928 18446744073709551615 4270679 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 22031 20543 132 53 0 20454 0
Current children cumulated CPU time (s) 3.08
Current children cumulated vsize (KiB) 88124

[startup+6.30052 s]
/proc/loadavg: 4.04 2.92 2.34 5/371 23476
/proc/meminfo: memFree=24904036/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=92496 CPUtime=6.27 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 48125 0 0 0 620 7 0 0 25 0 1 0 733251767 94715904 21672 33554432000 4194304 4409284 140736408988928 18446744073709551615 4324237 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 23124 21672 136 53 0 21547 0
Current children cumulated CPU time (s) 6.27
Current children cumulated vsize (KiB) 92496

[startup+12.7003 s]
/proc/loadavg: 4.04 2.94 2.35 5/371 23476
/proc/meminfo: memFree=24880956/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=131444 CPUtime=12.67 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 57868 0 0 0 1259 8 0 0 25 0 1 0 733251767 134598656 31415 33554432000 4194304 4409284 140736408988928 18446744073709551615 4245947 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 32861 31415 136 53 0 31284 0
Current children cumulated CPU time (s) 12.67
Current children cumulated vsize (KiB) 131444

[startup+25.501 s]
/proc/loadavg: 4.03 2.99 2.38 5/371 23477
/proc/meminfo: memFree=24870900/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=150808 CPUtime=25.47 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 60873 0 0 0 2538 9 0 0 25 0 1 0 733251767 154427392 34182 33554432000 4194304 4409284 140736408988928 18446744073709551615 4230341 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 37702 34182 141 53 0 36125 0
Current children cumulated CPU time (s) 25.47
Current children cumulated vsize (KiB) 150808

[startup+51.1012 s]
/proc/loadavg: 4.02 3.07 2.42 5/371 23477
/proc/meminfo: memFree=24824956/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=150804 CPUtime=51.08 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 63029 0 0 0 5096 12 0 0 25 0 1 0 733251767 154423296 36228 33554432000 4194304 4409284 140736408988928 18446744073709551615 4278576 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 37701 36228 141 53 0 36124 0
Current children cumulated CPU time (s) 51.08
Current children cumulated vsize (KiB) 150804

[startup+102.301 s]
/proc/loadavg: 4.01 3.21 2.50 5/371 23479
/proc/meminfo: memFree=24812752/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=158360 CPUtime=102.29 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 66185 0 0 0 10212 17 0 0 25 0 1 0 733251767 162160640 37094 33554432000 4194304 4409284 140736408988928 18446744073709551615 4231082 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 39590 37094 141 53 0 38013 0
Current children cumulated CPU time (s) 102.29
Current children cumulated vsize (KiB) 158360

[startup+162.301 s]
/proc/loadavg: 4.13 3.38 2.60 6/371 23480
/proc/meminfo: memFree=24811116/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=163976 CPUtime=162.28 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 67604 0 0 0 16207 21 0 0 25 0 1 0 733251767 167911424 38498 33554432000 4194304 4409284 140736408988928 18446744073709551615 4358699 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 40994 38498 141 53 0 39417 0
Current children cumulated CPU time (s) 162.28
Current children cumulated vsize (KiB) 163976

[startup+222.314 s]
/proc/loadavg: 4.05 3.49 2.68 5/371 23482
/proc/meminfo: memFree=24810720/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=157808 CPUtime=222.29 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 83990 0 0 0 22203 26 0 0 25 0 1 0 733251767 161595392 36956 33554432000 4194304 4409284 140736408988928 18446744073709551615 4359840 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 39452 36956 141 53 0 37875 0
Current children cumulated CPU time (s) 222.29
Current children cumulated vsize (KiB) 157808

[startup+282.3 s]
/proc/loadavg: 4.14 3.61 2.77 5/371 23484
/proc/meminfo: memFree=24798532/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=162668 CPUtime=282.28 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 101668 0 0 0 28197 31 0 0 25 0 1 0 733251767 166572032 38171 33554432000 4194304 4409284 140736408988928 18446744073709551615 4358371 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 40667 38171 141 53 0 39090 0
Current children cumulated CPU time (s) 282.28
Current children cumulated vsize (KiB) 162668

[startup+342.301 s]
/proc/loadavg: 4.05 3.68 2.85 6/371 23485
/proc/meminfo: memFree=24781768/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=157132 CPUtime=342.28 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 101668 0 0 0 34192 36 0 0 25 0 1 0 733251767 160903168 36787 33554432000 4194304 4409284 140736408988928 18446744073709551615 4230803 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 39283 36787 141 53 0 37706 0
Current children cumulated CPU time (s) 342.28
Current children cumulated vsize (KiB) 157132

[startup+402.301 s]
/proc/loadavg: 4.02 3.74 2.92 5/371 23487
/proc/meminfo: memFree=24801088/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=161332 CPUtime=402.28 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 119178 0 0 0 40187 41 0 0 25 0 1 0 733251767 165203968 37837 33554432000 4194304 4409284 140736408988928 18446744073709551615 4358212 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 40333 37837 141 53 0 38756 0
Current children cumulated CPU time (s) 402.28
Current children cumulated vsize (KiB) 161332

[startup+462.301 s]
/proc/loadavg: 4.00 3.78 2.98 5/371 23488
/proc/meminfo: memFree=24765100/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=161332 CPUtime=462.28 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 119178 0 0 0 46182 46 0 0 25 0 1 0 733251767 165203968 37837 33554432000 4194304 4409284 140736408988928 18446744073709551615 4230794 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 40333 37837 141 53 0 38756 0
Current children cumulated CPU time (s) 462.28
Current children cumulated vsize (KiB) 161332

[startup+522.301 s]
/proc/loadavg: 4.00 3.82 3.04 5/371 23492
/proc/meminfo: memFree=24750184/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=156972 CPUtime=522.28 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 119245 0 0 0 52176 52 0 0 25 0 1 0 733251767 160739328 36747 33554432000 4194304 4409284 140736408988928 18446744073709551615 4271337 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 39243 36747 141 53 0 37666 0
Current children cumulated CPU time (s) 522.28
Current children cumulated vsize (KiB) 156972

[startup+582.301 s]
/proc/loadavg: 4.00 3.85 3.10 5/371 23495
/proc/meminfo: memFree=24780596/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=160508 CPUtime=582.28 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 136584 0 0 0 58171 57 0 0 25 0 1 0 733251767 164360192 37631 33554432000 4194304 4409284 140736408988928 18446744073709551615 4358418 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 40127 37631 141 53 0 38550 0
Current children cumulated CPU time (s) 582.28
Current children cumulated vsize (KiB) 160508

[startup+642.301 s]
/proc/loadavg: 4.00 3.87 3.15 5/371 23496
/proc/meminfo: memFree=24758624/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=160508 CPUtime=642.28 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 136584 0 0 0 64165 63 0 0 25 0 1 0 733251767 164360192 37631 33554432000 4194304 4409284 140736408988928 18446744073709551615 4244764 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 40127 37631 141 53 0 38550 0
Current children cumulated CPU time (s) 642.28
Current children cumulated vsize (KiB) 160508

[startup+702.301 s]
/proc/loadavg: 4.07 3.92 3.21 5/371 23498
/proc/meminfo: memFree=24755376/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=160504 CPUtime=702.28 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 154052 0 0 0 70158 70 0 0 25 0 1 0 733251767 164356096 38653 33554432000 4194304 4409284 140736408988928 18446744073709551615 4358619 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 40126 38653 141 53 0 38549 0
Current children cumulated CPU time (s) 702.28
Current children cumulated vsize (KiB) 160504

[startup+762.3 s]
/proc/loadavg: 4.15 3.98 3.28 5/371 23499
/proc/meminfo: memFree=24701160/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=156100 CPUtime=762.28 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 154052 0 0 0 76153 75 0 0 25 0 1 0 733251767 159846400 37552 33554432000 4194304 4409284 140736408988928 18446744073709551615 251367571351 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 39025 37552 141 53 0 37448 0
Current children cumulated CPU time (s) 762.28
Current children cumulated vsize (KiB) 156100

[startup+822.3 s]
/proc/loadavg: 4.05 3.98 3.32 5/371 23501
/proc/meminfo: memFree=24749116/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=159308 CPUtime=822.27 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 154856 0 0 0 82148 79 0 0 25 0 1 0 733251767 163131392 38354 33554432000 4194304 4409284 140736408988928 18446744073709551615 4358519 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 39827 38354 141 53 0 38250 0
Current children cumulated CPU time (s) 822.27
Current children cumulated vsize (KiB) 159308

[startup+882.314 s]
/proc/loadavg: 4.06 3.99 3.36 5/371 23503
/proc/meminfo: memFree=24733328/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=159556 CPUtime=882.28 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 154918 0 0 0 88144 84 0 0 25 0 1 0 733251767 163385344 38416 33554432000 4194304 4409284 140736408988928 18446744073709551615 4269759 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 39889 38416 141 53 0 38312 0
Current children cumulated CPU time (s) 882.28
Current children cumulated vsize (KiB) 159556

[startup+942.301 s]
/proc/loadavg: 4.02 3.99 3.40 5/371 23504
/proc/meminfo: memFree=24779424/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=159308 CPUtime=942.27 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 171304 0 0 0 94136 91 0 0 25 0 1 0 733251767 163131392 38354 33554432000 4194304 4409284 140736408988928 18446744073709551615 4360278 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 39827 38354 141 53 0 38250 0
Current children cumulated CPU time (s) 942.27
Current children cumulated vsize (KiB) 159308

[startup+1002.3 s]
/proc/loadavg: 4.01 3.99 3.43 5/371 23506
/proc/meminfo: memFree=24770352/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=158976 CPUtime=1002.27 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 172067 0 0 0 100130 97 0 0 25 0 1 0 733251767 162791424 38271 33554432000 4194304 4409284 140736408988928 18446744073709551615 4229151 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 39744 38271 141 53 0 38167 0
Current children cumulated CPU time (s) 1002.27
Current children cumulated vsize (KiB) 158976

[startup+1062.3 s]
/proc/loadavg: 4.00 3.99 3.47 5/371 23507
/proc/meminfo: memFree=24779876/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=159216 CPUtime=1062.27 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 172128 0 0 0 106125 102 0 0 25 0 1 0 733251767 163037184 38331 33554432000 4194304 4409284 140736408988928 18446744073709551615 4358418 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 39804 38331 141 53 0 38227 0
Current children cumulated CPU time (s) 1062.27
Current children cumulated vsize (KiB) 159216

[startup+1122.3 s]
/proc/loadavg: 4.00 3.99 3.50 5/371 23509
/proc/meminfo: memFree=24774264/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=158972 CPUtime=1122.28 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 204898 0 0 0 112117 111 0 0 25 0 1 0 733251767 162787328 38270 33554432000 4194304 4409284 140736408988928 18446744073709551615 4358396 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 39743 38270 141 53 0 38166 0
Current children cumulated CPU time (s) 1122.28
Current children cumulated vsize (KiB) 158972

[startup+1182.3 s]
/proc/loadavg: 4.06 4.00 3.53 5/371 23622
/proc/meminfo: memFree=24875880/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=158972 CPUtime=1182.27 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 204898 0 0 0 118113 114 0 0 25 0 1 0 733251767 162787328 38270 33554432000 4194304 4409284 140736408988928 18446744073709551615 4358433 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 39743 38270 141 53 0 38166 0
Current children cumulated CPU time (s) 1182.27
Current children cumulated vsize (KiB) 158972

[startup+1242.3 s]
/proc/loadavg: 4.08 4.01 3.56 5/371 23623
/proc/meminfo: memFree=24858812/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=158972 CPUtime=1242.27 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 204898 0 0 0 124106 121 0 0 25 0 1 0 733251767 162787328 38270 33554432000 4194304 4409284 140736408988928 18446744073709551615 4358371 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 39743 38270 141 53 0 38166 0
Current children cumulated CPU time (s) 1242.27
Current children cumulated vsize (KiB) 158972



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1300 s]
/proc/loadavg: 4.03 4.01 3.58 5/371 23625
/proc/meminfo: memFree=24823440/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=159196 CPUtime=1299.98 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 204956 0 0 0 129872 126 0 0 25 0 1 0 733251767 163016704 38326 33554432000 4194304 4409284 140736408988928 18446744073709551615 4256901 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 39799 38326 141 53 0 38222 0
Current children cumulated CPU time (s) 1299.98
Current children cumulated vsize (KiB) 159196

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1300 s]
/proc/loadavg: 4.03 4.01 3.58 5/371 23625
/proc/meminfo: memFree=24823440/32950928 swapFree=38710892/67111528
[pid=23475] ppid=23473 vsize=159196 CPUtime=1299.98 cores=0,2
/proc/23475/stat : 23475 (lingeling) R 23473 23475 23384 0 -1 4202496 204956 0 0 0 129872 126 0 0 25 0 1 0 733251767 163016704 38326 33554432000 4194304 4409284 140736408988928 18446744073709551615 4256901 0 0 4096 17506 0 0 0 17 0 0 0 0
/proc/23475/statm: 39799 38326 141 53 0 38222 0
Current children cumulated CPU time (s) 1299.98
Current children cumulated vsize (KiB) 159196

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1300.02
CPU time (s): 1300
CPU user time (s): 1298.73
CPU system time (s): 1.26881
CPU usage (%): 99.9986
Max. virtual memory (cumulated for all children) (KiB): 224756

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1298.73
system time used= 1.26881
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 204956
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 4
involuntary context switches= 4202

runsolver used 4.95925 second user time and 10.4054 second system time

The end

Launcher Data 

Begin job on node113 at 2011-05-15 05:30:11
IDJOB=3371847
IDBENCH=20410
IDSOLVER=1849
FILE ID=node113/3371847-1305430210
RUNJOBID= node113-1305430043-23402
PBS_JOBID= 13324617
Free space on /tmp= 73436 MiB

SOLVER NAME= Lingeling 587f (fixed)
BENCH NAME= SAT07/industrial/palacios/hard/sortnet-8-ipc5-h19-sat.cnf
COMMAND LINE= lingeling BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3371847-1305430210/watcher-3371847-1305430210 -o /tmp/evaluation-result-3371847-1305430210/solver-3371847-1305430210 -C 2400 -W 1300 -M 7680  lingeling HOME/instance-3371847-1305430210.cnf

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 7680 MiB
NBCORE= 2

MD5SUM BENCH= 409f4401e9e8b60ee665ac5567a13e41
RANDOM SEED=1790222161

node113.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      25018176 kB
Buffers:        263240 kB
Cached:        2867452 kB
SwapCached:      93712 kB
Active:         974716 kB
Inactive:      6693496 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      25018176 kB
SwapTotal:    67111528 kB
SwapFree:     38710892 kB
Dirty:           22964 kB
Writeback:           0 kB
AnonPages:     4455692 kB
Mapped:          14728 kB
Slab:           106604 kB
PageTables:      92144 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 87256460 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73444 MiB
End job on node113 at 2011-05-15 05:51:53