Trace number 1563020

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth-5-0 2009-03-19? (TO) 1200.23 608.012

General information on the benchmark

NameAPPLICATIONS/bioinfo/
q_query_3_l39_lambda.cnf
MD5SUMcc949593da8c10735a1b7ac8b30d4e4d
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark1.06784
Satisfiable
(Un)Satisfiability was proved
Number of variables27735
Number of clauses136449
Sum of the clauses size392486
Maximum clause length9
Minimum clause length1
Number of clauses of size 13722
Number of clauses of size 235083
Number of clauses of size 378429
Number of clauses of size 413232
Number of clauses of size 55866
Number of clauses of size over 5117

Solver Data 

0.39/0.40	c Command: HOME/ttsth-5-0 -num-threads 2 HOME/instance-1563020-1240999572.cnf 
0.39/0.40	c Using SATTIMEOUT of 1200
0.39/0.40	c Using SATRAM of 1800
0.39/0.40	c Size of subset node = 16
0.39/0.40	c Variables = 27735
0.39/0.40	c Clauses = 136449
0.39/0.40	c Literals = 392486
0.39/0.40	c Reduced size:
0.39/0.40	c Variables = 27735
0.39/0.40	c Clauses = 122058
0.39/0.40	c Literals = 356327
0.39/0.40	c a2
0.79/0.83	c a1
11.09/11.19	c Independent sub-problems = 1
11.09/11.19	c Sub-problem with 23105 variables and 122058 clauses
11.09/11.19	c using permutation method 4
11.09/11.19	c count = 1
11.09/11.19	c Ordering finished 11.120 seconds
11.09/11.19	c mapped 11.190 seconds
11.09/11.19	c permuted 11.190 seconds
11.09/11.19	c about to sort
11.19/11.24	c sorted
12.88/12.94	c clauses now 122058
12.88/12.94	c init finished 11.350 seconds
12.88/12.94	c tree built 12.930 seconds
12.88/12.94	c 
12.88/12.94	c PA progress                      | CPU progress Partial assignments | improvements      CPU time        PA/sec        nodes
12.88/12.94	c ---------------------------------|----------------------------------|------------------------------------------------------
12.88/12.94	c ................................ | ..........                       |                       12.9           0.0             
12.88/12.94	c ................................ | ..........                     1 |                       12.9           0.1             
12.88/12.94	c ****************................ | ..........                     2 |                       12.9           0.2            1
12.88/12.94	c ****************................ | ..........                     3 |                       12.9           0.2            1
12.88/12.94	c ****************................ | ..........                     4 |                       12.9           0.3            1
12.88/12.94	c ****************................ | ..........                     5 |                       12.9           0.4            1
12.88/12.94	c ****************................ | ..........                     6 |                       12.9           0.5            1
12.88/12.95	c ****************................ | ..........                  1000 | 3                     12.9          77.2          123
23.82/18.50	c ****************................ | ..........                172000 | 6681                  24.0        7157.7      2561293
47.97/30.56	c ****************................ | ..........                461000 | 17532                 48.1        9584.2      6640389
72.11/42.60	c ****************................ | ..........                647000 | 25545                 72.1        8968.7     10285614
96.06/54.63	c ****************................ | ..........                773000 | 30673                 96.1        8040.4     13193888
120.01/66.61	c ****************................ | *.........                887000 | 35233                120.0        7388.6     15646786
143.95/78.67	c ****************................ | *.........               1002000 | 39731                144.1        6952.1     17725624
167.89/90.67	c ****************................ | *.........               1113000 | 43800                168.1        6622.2     19739313
192.04/102.75	c ****************................ | *.........               1211000 | 47817                192.2        6301.4     21511414
215.79/114.70	c ****************................ | *.........               1320000 | 52077                216.0        6110.3     23266845
239.93/126.76	c ****************................ | **........               1415000 | 55584                240.1        5893.4     24667135
263.89/138.79	c ****************................ | **........               1517000 | 59157                264.1        5743.6     26260601
288.23/150.95	c ****************................ | **........               1611000 | 62526                288.4        5586.2     28073633
312.17/162.90	c ****************................ | **........               1719000 | 66128                312.2        5505.6     29992430
335.93/174.86	c ****************................ | **........               1806000 | 69263                336.1        5373.4     31627654
359.87/186.86	c ****************................ | ***.......               1891000 | 72273                360.1        5251.9     33248415
384.02/198.97	c ****************................ | ***.......               1963000 | 74869                384.2        5109.1     34661915
408.16/211.03	c ****************................ | ***.......               2036000 | 77749                408.3        4986.8     36268533
431.91/222.96	c ****************................ | ***.......               2123000 | 80827                432.1        4913.2     37945220
456.06/235.01	c ****************................ | ***.......               2255000 | 85586                456.2        4943.4     39838969
479.81/246.97	c ****************................ | ****......               2357000 | 90215                480.0        4910.1     41292954
503.95/259.08	c ****************................ | ****......               2455000 | 94676                504.2        4869.0     43049288
527.89/271.07	c ****************................ | ****......               2526000 | 97487                528.1        4783.0     44417301
552.04/283.14	c ****************................ | ****......               2611000 | 101035               552.2        4728.2     46369690
575.99/295.20	c ****************................ | ****......               2692000 | 104341               576.3        4671.3     48649671
599.94/307.20	c ****************................ | *****.....               2767000 | 107473               600.2        4609.9     50690121
624.09/319.20	c ****************................ | *****.....               2836000 | 110087               624.2        4543.4     52851937
648.03/331.25	c ****************................ | *****.....               2907000 | 112740               648.2        4484.5     54716588
672.17/343.30	c ****************................ | *****.....               2977000 | 115368               672.3        4428.2     56303014
695.94/355.24	c ****************................ | *****.....               3047000 | 118000               696.1        4377.1     57735175
719.88/367.28	c ****************................ | ******....               3117000 | 120674               720.1        4328.3     59198629
744.02/379.33	c ****************................ | ******....               3181000 | 123151               744.2        4274.3     60526441
767.96/391.35	c ****************................ | ******....               3254000 | 126049               768.2        4236.0     61935983
791.91/403.34	c ****************................ | ******....               3367000 | 129723               792.1        4250.7     63722825
815.85/415.37	c ****************................ | ******....               3474000 | 133806               816.1        4256.7     65661829
839.80/427.38	c ****************................ | *******...               3569000 | 137378               840.1        4248.3     67504427
863.95/439.45	c ****************................ | *******...               3651000 | 141440               864.2        4224.8     69003819
887.89/451.48	c ****************................ | *******...               3712000 | 143563               888.2        4179.3     70185992
912.25/463.60	c ****************................ | *******...               3775000 | 147322               912.4        4137.5     71485977
935.79/475.45	c ****************................ | *******...               3850000 | 153084               936.0        4113.0     72861533
959.94/487.56	c ****************................ | ********..               3932000 | 156701               960.2        4094.9     74168244
984.09/499.66	c ****************................ | ********..               4004000 | 159039               984.4        4067.6     75235751
1008.03/511.64	c ****************................ | ********..               4072000 | 161609              1008.3        4038.6     76278274
1031.98/523.61	c ****************................ | ********..               4138000 | 165093              1032.2        4009.1     77198634
1055.72/535.60	c ****************................ | ********..               4200000 | 168399              1056.1        3977.0     78028285
1080.08/547.77	c ****************................ | *********.               4260000 | 170775              1080.4        3943.1     79244923
1103.82/559.64	c ****************................ | *********.               4306000 | 172532              1104.1        3900.1     80192409
1128.16/571.83	c ****************................ | *********.               4362000 | 174688              1128.4        3865.7     81164934
1152.31/583.91	c ****************................ | *********.               4416000 | 176900              1152.5        3831.7     82151008
1175.85/595.77	c ****************................ | *********.               4473000 | 179071              1176.2        3803.0     83056940

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver version 3.2.9 (svn:492) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-1563020-1240999572/watcher-1563020-1240999572 -o /tmp/evaluation-result-1563020-1240999572/solver-1563020-1240999572 -C 1200 -W 1800 -M 1800 HOME/ttsth-5-0 -num-threads 2 HOME/instance-1563020-1240999572.cnf 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1200 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1230 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 1843200 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 1894400 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.93 0.98 1.01 3/64 7755
/proc/meminfo: memFree=1409152/2055920 swapFree=4179744/4192956
[pid=7755] ppid=7753 vsize=1064 CPUtime=0
/proc/7755/stat : 7755 (ttsth-5-0) R 7753 7755 7567 0 -1 4194304 146 0 0 0 0 0 0 0 18 0 1 0 127983397 1089536 130 1992294400 134512640 135237372 4294956192 18446744073709551615 134732614 0 0 4096 0 0 0 0 17 1 0 0
/proc/7755/statm: 266 130 77 176 0 88 0

[startup+0.042604 s]
/proc/loadavg: 0.93 0.98 1.01 3/64 7755
/proc/meminfo: memFree=1409152/2055920 swapFree=4179744/4192956
[pid=7755] ppid=7753 vsize=2412 CPUtime=0.04
/proc/7755/stat : 7755 (ttsth-5-0) R 7753 7755 7567 0 -1 4194304 476 0 0 0 4 0 0 0 18 0 1 0 127983397 2469888 460 1992294400 134512640 135237372 4294956192 18446744073709551615 134732614 0 0 4096 0 0 0 0 17 1 0 0
/proc/7755/statm: 603 461 78 176 0 425 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 2412

[startup+0.101614 s]
/proc/loadavg: 0.93 0.98 1.01 3/64 7755
/proc/meminfo: memFree=1409152/2055920 swapFree=4179744/4192956
[pid=7755] ppid=7753 vsize=4372 CPUtime=0.09
/proc/7755/stat : 7755 (ttsth-5-0) R 7753 7755 7567 0 -1 4194304 978 0 0 0 9 0 0 0 18 0 1 0 127983397 4476928 962 1992294400 134512640 135237372 4294956192 18446744073709551615 134732614 0 0 4096 0 0 0 0 17 1 0 0
/proc/7755/statm: 1093 962 78 176 0 915 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 4372

[startup+0.30165 s]
/proc/loadavg: 0.93 0.98 1.01 3/64 7755
/proc/meminfo: memFree=1409152/2055920 swapFree=4179744/4192956
[pid=7755] ppid=7753 vsize=10220 CPUtime=0.29
/proc/7755/stat : 7755 (ttsth-5-0) R 7753 7755 7567 0 -1 4194304 2400 0 0 0 28 1 0 0 19 0 1 0 127983397 10465280 2384 1992294400 134512640 135237372 4294956192 18446744073709551615 134534364 0 0 4096 0 0 0 0 17 1 0 0
/proc/7755/statm: 2555 2384 78 176 0 2377 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 10220

[startup+0.701722 s]
/proc/loadavg: 0.93 0.98 1.01 3/64 7755
/proc/meminfo: memFree=1409152/2055920 swapFree=4179744/4192956
[pid=7755] ppid=7753 vsize=19512 CPUtime=0.69
/proc/7755/stat : 7755 (ttsth-5-0) R 7753 7755 7567 0 -1 4194304 4648 0 0 0 67 2 0 0 22 0 1 0 127983397 19980288 4631 1992294400 134512640 135237372 4294956192 18446744073709551615 134680257 0 0 4096 0 0 0 0 17 1 0 0
/proc/7755/statm: 4878 4631 79 176 0 4700 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 19512

[startup+1.50187 s]
/proc/loadavg: 0.94 0.98 1.00 2/65 7756
/proc/meminfo: memFree=1390904/2055920 swapFree=4179744/4192956
[pid=7755] ppid=7753 vsize=23188 CPUtime=1.49
/proc/7755/stat : 7755 (ttsth-5-0) R 7753 7755 7567 0 -1 4194304 6011 0 0 0 147 2 0 0 25 0 1 0 127983397 23744512 5626 1992294400 134512640 135237372 4294956192 18446744073709551615 134555279 0 0 4096 0 0 0 0 17 1 0 0
/proc/7755/statm: 5797 5626 80 176 0 5619 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 23188

[startup+3.10116 s]
/proc/loadavg: 0.94 0.98 1.00 2/65 7756
/proc/meminfo: memFree=1386744/2055920 swapFree=4179744/4192956
[pid=7755] ppid=7753 vsize=23544 CPUtime=3.08
/proc/7755/stat : 7755 (ttsth-5-0) R 7753 7755 7567 0 -1 4194304 6051 0 0 0 306 2 0 0 25 0 1 0 127983397 24109056 5666 1992294400 134512640 135237372 4294956192 18446744073709551615 134600978 0 0 4096 0 0 0 0 17 1 0 0
/proc/7755/statm: 5886 5666 80 176 0 5708 0
Current children cumulated CPU time (s) 3.08
Current children cumulated vsize (KiB) 23544

[startup+6.30273 s]
/proc/loadavg: 0.94 0.98 1.00 2/65 7756
/proc/meminfo: memFree=1386424/2055920 swapFree=4179744/4192956
[pid=7755] ppid=7753 vsize=23544 CPUtime=6.29
/proc/7755/stat : 7755 (ttsth-5-0) R 7753 7755 7567 0 -1 4194304 6092 0 0 0 627 2 0 0 25 0 1 0 127983397 24109056 5707 1992294400 134512640 135237372 4294956192 18446744073709551615 134600966 0 0 4096 0 0 0 0 17 1 0 0
/proc/7755/statm: 5886 5707 80 176 0 5708 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 23544

[startup+12.7019 s]
/proc/loadavg: 0.95 0.98 1.00 2/65 7756
/proc/meminfo: memFree=1357816/2055920 swapFree=4179744/4192956
[pid=7755] ppid=7753 vsize=76388 CPUtime=12.69
/proc/7755/stat : 7755 (ttsth-5-0) R 7753 7755 7567 0 -1 4194304 19557 0 0 0 1261 8 0 0 25 0 1 0 127983397 78221312 18932 1992294400 134512640 135237372 4294956192 18446744073709551615 134551487 0 0 4096 0 0 0 0 17 1 0 0
/proc/7755/statm: 19097 18932 94 176 0 18919 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 76388

[startup+25.5022 s]
/proc/loadavg: 1.11 1.02 1.01 3/67 7758
/proc/meminfo: memFree=1250664/2055920 swapFree=4179744/4192956
[pid=7755] ppid=7753 vsize=183616 CPUtime=37.99
/proc/7755/stat : 7755 (ttsth-5-0) S 7753 7755 7567 0 -1 4194304 40703 0 0 0 3780 19 0 0 25 0 3 0 127983397 188022784 40078 1992294400 134512640 135237372 4294956192 18446744073709551615 134629544 0 0 4096 0 18446744073709551615 0 0 17 1 0 0
/proc/7755/statm: 45904 40078 98 176 0 45726 0
[pid=7755/tid=7757] ppid=7753 vsize=183616 CPUtime=12.55
/proc/7755/task/7757/stat : 7757 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 11383 0 0 0 1249 6 0 0 25 0 3 0 127984691 188022784 40078 1992294400 134512640 135237372 4294956192 18446744073709551615 134680502 0 0 4096 0 0 0 0 -1 1 0 0
[pid=7755/tid=7758] ppid=7753 vsize=183616 CPUtime=12.5
/proc/7755/task/7758/stat : 7758 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 9428 0 0 0 1245 5 0 0 25 0 3 0 127984691 188022784 40078 1992294400 134512640 135237372 4294956192 18446744073709551615 134680491 0 0 4096 0 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 37.99
Current children cumulated vsize (KiB) 183616

[startup+51.1028 s]
/proc/loadavg: 1.41 1.09 1.04 3/67 7758
/proc/meminfo: memFree=1136872/2055920 swapFree=4179744/4192956
[pid=7755] ppid=7753 vsize=302000 CPUtime=89.08
/proc/7755/stat : 7755 (ttsth-5-0) S 7753 7755 7567 0 -1 4194304 69160 0 0 0 8873 35 0 0 25 0 3 0 127983397 309248000 68535 1992294400 134512640 135237372 4294956192 18446744073709551615 134629544 0 0 4096 0 18446744073709551615 0 0 17 1 0 0
/proc/7755/statm: 75500 68535 98 176 0 75322 0
[pid=7755/tid=7757] ppid=7753 vsize=302000 CPUtime=38.14
/proc/7755/task/7757/stat : 7757 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 26724 0 0 0 3801 13 0 0 25 0 3 0 127984691 309248000 68535 1992294400 134512640 135237372 4294956192 18446744073709551615 134562341 0 0 4096 0 0 0 0 -1 1 0 0
[pid=7755/tid=7758] ppid=7753 vsize=302000 CPUtime=37.99
/proc/7755/task/7758/stat : 7758 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 22544 0 0 0 3787 12 0 0 25 0 3 0 127984691 309248000 68535 1992294400 134512640 135237372 4294956192 18446744073709551615 134562338 0 0 4096 0 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 89.08
Current children cumulated vsize (KiB) 302000

[startup+102.306 s]
/proc/loadavg: 1.76 1.24 1.09 3/67 7758
/proc/meminfo: memFree=994152/2055920 swapFree=4179744/4192956
[pid=7755] ppid=7753 vsize=448240 CPUtime=191.25
/proc/7755/stat : 7755 (ttsth-5-0) S 7753 7755 7567 0 -1 4194304 104274 0 0 0 19069 56 0 0 25 0 3 0 127983397 458997760 103649 1992294400 134512640 135237372 4294956192 18446744073709551615 134629544 0 0 4096 0 18446744073709551615 0 0 17 1 0 0
/proc/7755/statm: 112060 103649 98 176 0 111882 0
[pid=7755/tid=7757] ppid=7753 vsize=448240 CPUtime=89.32
/proc/7755/task/7757/stat : 7757 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 44059 0 0 0 8909 23 0 0 25 0 3 0 127984691 458997760 103649 1992294400 134512640 135237372 4294956192 18446744073709551615 134562338 0 0 4096 0 0 0 0 -1 1 0 0
[pid=7755/tid=7758] ppid=7753 vsize=448240 CPUtime=88.99
/proc/7755/task/7758/stat : 7758 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 40323 0 0 0 8875 24 0 0 25 0 3 0 127984691 458997760 103649 1992294400 134512640 135237372 4294956192 18446744073709551615 134590707 0 0 4096 0 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 191.25
Current children cumulated vsize (KiB) 448240

[startup+162.302 s]
/proc/loadavg: 1.91 1.38 1.14 3/67 7758
/proc/meminfo: memFree=862504/2055920 swapFree=4179744/4192956
[pid=7755] ppid=7753 vsize=587476 CPUtime=310.98
/proc/7755/stat : 7755 (ttsth-5-0) S 7753 7755 7567 0 -1 4194304 137434 0 0 0 31020 78 0 0 25 0 3 0 127983397 601575424 136809 1992294400 134512640 135237372 4294956192 18446744073709551615 134629544 0 0 4096 0 18446744073709551615 0 0 17 1 0 0
/proc/7755/statm: 146869 136809 98 176 0 146691 0
[pid=7755/tid=7757] ppid=7753 vsize=587476 CPUtime=149.3
/proc/7755/task/7757/stat : 7757 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 61754 0 0 0 14897 33 0 0 25 0 3 0 127984691 601575424 136809 1992294400 134512640 135237372 4294956192 18446744073709551615 134562344 0 0 4096 0 0 0 0 -1 1 0 0
[pid=7755/tid=7758] ppid=7753 vsize=587476 CPUtime=148.74
/proc/7755/task/7758/stat : 7758 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 55788 0 0 0 14838 36 0 0 25 0 3 0 127984691 601575424 136809 1992294400 134512640 135237372 4294956192 18446744073709551615 134591065 0 0 4096 0 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 310.98
Current children cumulated vsize (KiB) 587476

[startup+222.302 s]
/proc/loadavg: 1.97 1.49 1.20 3/67 7758
/proc/meminfo: memFree=737704/2055920 swapFree=4179744/4192956
[pid=7755] ppid=7753 vsize=715520 CPUtime=430.72
/proc/7755/stat : 7755 (ttsth-5-0) S 7753 7755 7567 0 -1 4194304 168327 0 0 0 42975 97 0 0 25 0 3 0 127983397 732692480 167702 1992294400 134512640 135237372 4294956192 18446744073709551615 134629544 0 0 4096 0 18446744073709551615 0 0 17 1 0 0
/proc/7755/statm: 178880 167702 98 176 0 178702 0
[pid=7755/tid=7757] ppid=7753 vsize=715520 CPUtime=209.28
/proc/7755/task/7757/stat : 7757 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 76654 0 0 0 20887 41 0 0 25 0 3 0 127984691 732692480 167702 1992294400 134512640 135237372 4294956192 18446744073709551615 134562355 0 0 4096 0 0 0 0 -1 1 0 0
[pid=7755/tid=7758] ppid=7753 vsize=715520 CPUtime=208.5
/proc/7755/task/7758/stat : 7758 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 71781 0 0 0 20803 47 0 0 25 0 3 0 127984691 732692480 167702 1992294400 134512640 135237372 4294956192 18446744073709551615 134562341 0 0 4096 0 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 430.72
Current children cumulated vsize (KiB) 715520

[startup+282.302 s]
/proc/loadavg: 1.98 1.58 1.24 3/67 7758
/proc/meminfo: memFree=608232/2055920 swapFree=4179744/4192956
[pid=7755] ppid=7753 vsize=852300 CPUtime=550.45
/proc/7755/stat : 7755 (ttsth-5-0) S 7753 7755 7567 0 -1 4194304 201068 0 0 0 54925 120 0 0 25 0 3 0 127983397 872755200 200443 1992294400 134512640 135237372 4294956192 18446744073709551615 134629544 0 0 4096 0 18446744073709551615 0 0 17 1 0 0
/proc/7755/statm: 213075 200443 98 176 0 212897 0
[pid=7755/tid=7757] ppid=7753 vsize=852300 CPUtime=269.25
/proc/7755/task/7757/stat : 7757 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 96397 0 0 0 26872 53 0 0 25 0 3 0 127984691 872755200 200443 1992294400 134512640 135237372 4294956192 18446744073709551615 134562380 0 0 4096 0 0 0 0 -1 1 0 0
[pid=7755/tid=7758] ppid=7753 vsize=852300 CPUtime=268.25
/proc/7755/task/7758/stat : 7758 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 84779 0 0 0 26767 58 0 0 25 0 3 0 127984691 872755200 200443 1992294400 134512640 135237372 4294956192 18446744073709551615 134562328 0 0 4096 0 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 550.45
Current children cumulated vsize (KiB) 852300

[startup+342.301 s]
/proc/loadavg: 1.99 1.65 1.29 3/67 7758
/proc/meminfo: memFree=448744/2055920 swapFree=4179744/4192956
[pid=7755] ppid=7753 vsize=1013404 CPUtime=670.18
/proc/7755/stat : 7755 (ttsth-5-0) S 7753 7755 7567 0 -1 4194304 240302 0 0 0 66874 144 0 0 25 0 3 0 127983397 1037725696 239677 1992294400 134512640 135237372 4294956192 18446744073709551615 134629544 0 0 4096 0 18446744073709551615 0 0 17 1 0 0
/proc/7755/statm: 253351 239677 98 176 0 253173 0
[pid=7755/tid=7757] ppid=7753 vsize=1013404 CPUtime=329.23
/proc/7755/task/7757/stat : 7757 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 110052 0 0 0 32861 62 0 0 25 0 3 0 127984691 1037725696 239677 1992294400 134512640 135237372 4294956192 18446744073709551615 134562338 0 0 4096 0 0 0 0 -1 1 0 0
[pid=7755/tid=7758] ppid=7753 vsize=1013404 CPUtime=328.01
/proc/7755/task/7758/stat : 7758 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 110358 0 0 0 32728 73 0 0 25 0 3 0 127984691 1037725696 239677 1992294400 134512640 135237372 4294956192 18446744073709551615 134562338 0 0 4096 0 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 670.18
Current children cumulated vsize (KiB) 1013404

[startup+402.302 s]
/proc/loadavg: 1.99 1.71 1.33 3/67 7758
/proc/meminfo: memFree=335912/2055920 swapFree=4179744/4192956
[pid=7755] ppid=7753 vsize=1132676 CPUtime=789.92
/proc/7755/stat : 7755 (ttsth-5-0) S 7753 7755 7567 0 -1 4194304 269031 0 0 0 78832 160 0 0 25 0 3 0 127983397 1159860224 268406 1992294400 134512640 135237372 4294956192 18446744073709551615 134629544 0 0 4096 0 18446744073709551615 0 0 17 1 0 0
/proc/7755/statm: 283169 268406 98 176 0 282991 0
[pid=7755/tid=7757] ppid=7753 vsize=1132676 CPUtime=389.2
/proc/7755/task/7757/stat : 7757 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 124480 0 0 0 38851 69 0 0 25 0 3 0 127984691 1159860224 268406 1992294400 134512640 135237372 4294956192 18446744073709551615 134562338 0 0 4096 0 0 0 0 -1 1 0 0
[pid=7755/tid=7758] ppid=7753 vsize=1132676 CPUtime=387.76
/proc/7755/task/7758/stat : 7758 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 124659 0 0 0 38695 81 0 0 25 0 3 0 127984691 1159860224 268406 1992294400 134512640 135237372 4294956192 18446744073709551615 134562351 0 0 4096 0 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 789.92
Current children cumulated vsize (KiB) 1132676

[startup+462.301 s]
/proc/loadavg: 1.99 1.76 1.37 3/67 7758
/proc/meminfo: memFree=211432/2055920 swapFree=4179744/4192956
[pid=7755] ppid=7753 vsize=1261668 CPUtime=909.65
/proc/7755/stat : 7755 (ttsth-5-0) S 7753 7755 7567 0 -1 4194304 299545 0 0 0 90785 180 0 0 25 0 3 0 127983397 1291948032 298920 1992294400 134512640 135237372 4294956192 18446744073709551615 134629544 0 0 4096 0 18446744073709551615 0 0 17 1 0 0
/proc/7755/statm: 315417 298920 98 176 0 315239 0
[pid=7755/tid=7757] ppid=7753 vsize=1261668 CPUtime=449.18
/proc/7755/task/7757/stat : 7757 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 144383 0 0 0 44837 81 0 0 25 0 3 0 127984691 1291948032 298920 1992294400 134512640 135237372 4294956192 18446744073709551615 134562338 0 0 4096 0 0 0 0 -1 1 0 0
[pid=7755/tid=7758] ppid=7753 vsize=1261668 CPUtime=447.52
/proc/7755/task/7758/stat : 7758 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 135270 0 0 0 44662 90 0 0 25 0 3 0 127984691 1291948032 298920 1992294400 134512640 135237372 4294956192 18446744073709551615 134562380 0 0 4096 0 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 909.65
Current children cumulated vsize (KiB) 1261668

[startup+522.302 s]
/proc/loadavg: 1.99 1.80 1.40 3/67 7758
/proc/meminfo: memFree=121512/2055920 swapFree=4179744/4192956
[pid=7755] ppid=7753 vsize=1357140 CPUtime=1029.38
/proc/7755/stat : 7755 (ttsth-5-0) S 7753 7755 7567 0 -1 4194304 322063 0 0 0 102744 194 0 0 25 0 3 0 127983397 1389711360 321438 1992294400 134512640 135237372 4294956192 18446744073709551615 134629544 0 0 4096 0 18446744073709551615 0 0 17 1 0 0
/proc/7755/statm: 339285 321438 98 176 0 339107 0
[pid=7755/tid=7757] ppid=7753 vsize=1357140 CPUtime=509.16
/proc/7755/task/7757/stat : 7757 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 159875 0 0 0 50827 89 0 0 25 0 3 0 127984691 1389711360 321438 1992294400 134512640 135237372 4294956192 18446744073709551615 134546026 0 0 4096 0 0 0 0 -1 1 0 0
[pid=7755/tid=7758] ppid=7753 vsize=1357140 CPUtime=507.28
/proc/7755/task/7758/stat : 7758 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 142296 0 0 0 50632 96 0 0 25 0 3 0 127984691 1389711360 321438 1992294400 134512640 135237372 4294956192 18446744073709551615 134562300 0 0 4096 0 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 1029.38
Current children cumulated vsize (KiB) 1357140

[startup+582.302 s]
/proc/loadavg: 1.99 1.84 1.44 3/67 7758
/proc/meminfo: memFree=43240/2055920 swapFree=4179744/4192956
[pid=7755] ppid=7753 vsize=1438428 CPUtime=1149.11
/proc/7755/stat : 7755 (ttsth-5-0) S 7753 7755 7567 0 -1 4194304 341430 0 0 0 114705 206 0 0 25 0 3 0 127983397 1472950272 340805 1992294400 134512640 135237372 4294956192 18446744073709551615 134629544 0 0 4096 0 18446744073709551615 0 0 17 1 0 0
/proc/7755/statm: 359607 340805 98 176 0 359429 0
[pid=7755/tid=7757] ppid=7753 vsize=1438428 CPUtime=569.13
/proc/7755/task/7757/stat : 7757 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 171974 0 0 0 56818 95 0 0 25 0 3 0 127984691 1472950272 340805 1992294400 134512640 135237372 4294956192 18446744073709551615 134562338 0 0 4096 0 0 0 0 -1 1 0 0
[pid=7755/tid=7758] ppid=7753 vsize=1438428 CPUtime=567.04
/proc/7755/task/7758/stat : 7758 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 149564 0 0 0 56602 102 0 0 21 0 3 0 127984691 1472950272 340805 1992294400 134512640 135237372 4294956192 18446744073709551615 134562338 0 0 4096 0 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 1149.11
Current children cumulated vsize (KiB) 1438428



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+607.801 s]
/proc/loadavg: 1.99 1.85 1.45 3/67 7758
/proc/meminfo: memFree=16360/2055920 swapFree=4179744/4192956
[pid=7755] ppid=7753 vsize=1469916 CPUtime=1200
/proc/7755/stat : 7755 (ttsth-5-0) S 7753 7755 7567 0 -1 4194304 348806 0 0 0 119789 211 0 0 25 0 3 0 127983397 1505193984 348181 1992294400 134512640 135237372 4294956192 18446744073709551615 134629544 0 0 4096 0 18446744073709551615 0 0 17 1 0 0
/proc/7755/statm: 367479 348181 98 176 0 367301 0
[pid=7755/tid=7757] ppid=7753 vsize=1469916 CPUtime=594.63
/proc/7755/task/7757/stat : 7757 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 176339 0 0 0 59364 99 0 0 25 0 3 0 127984691 1505193984 348181 1992294400 134512640 135237372 4294956192 18446744073709551615 134562338 0 0 4096 0 0 0 0 -1 1 0 0
[pid=7755/tid=7758] ppid=7753 vsize=1469916 CPUtime=592.43
/proc/7755/task/7758/stat : 7758 (ttsth-5-0) R 7753 7755 7567 0 -1 4194368 152575 0 0 0 59140 103 0 0 25 0 3 0 127984691 1505193984 348181 1992294400 134512640 135237372 4294956192 18446744073709551615 134562338 0 0 4096 0 0 0 0 -1 0 0 0
Current children cumulated CPU time (s) 1200
Current children cumulated vsize (KiB) 1469916

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+608.002 s]
/proc/loadavg: 1.99 1.85 1.45 3/67 7758
/proc/meminfo: memFree=16360/2055920 swapFree=4179744/4192956
[pid=7755] ppid=7753 vsize=0 CPUtime=1200.2
/proc/7755/stat : 7755 (ttsth-5-0) Z 7753 7755 7567 0 -1 4195340 348808 0 0 0 119790 230 0 0 16 0 2 0 127983397 0 0 1992294400 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0
/proc/7755/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 1200.2
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 608.012
CPU time (s): 1200.23
CPU user time (s): 1197.91
CPU system time (s): 2.31565
CPU usage (%): 197.402
Max. virtual memory (cumulated for all children) (KiB): 1469916

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1197.91
system time used= 2.31565
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 348808
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 129
involuntary context switches= 9129

runsolver used 0.571913 second user time and 1.64975 second system time

The end

Launcher Data 

Begin job on node67 at 2009-04-29 12:06:12
IDJOB=1563020
IDBENCH=70945
IDSOLVER=549
FILE ID=node67/1563020-1240999572
PBS_JOBID= 9204804
Free space on /tmp= 66188 MiB

SOLVER NAME= ttsth-5-0 2009-03-19
BENCH NAME= SAT09/APPLICATIONS/bioinfo/q_query_3_l39_lambda.cnf
COMMAND LINE= HOME/ttsth-5-0 -num-threads MAXNBTHREAD BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-1563020-1240999572/watcher-1563020-1240999572 -o /tmp/evaluation-result-1563020-1240999572/solver-1563020-1240999572 -C 1200 -W 1800 -M 1800  HOME/ttsth-5-0 -num-threads 2 HOME/instance-1563020-1240999572.cnf

TIME LIMIT= 1200 seconds
MEMORY LIMIT= 1800 MiB
MAX NB THREAD= 2

MD5SUM BENCH= cc949593da8c10735a1b7ac8b30d4e4d
RANDOM SEED=270868383

node67.alineos.net Linux 2.6.9-22.EL.rootsmp #1 SMP Mon Oct 3 08:59:52 CEST 2005

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.277
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5914.62
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.277
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5586.94
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:      2055920 kB
MemFree:       1409568 kB
Buffers:         43808 kB
Cached:         550024 kB
SwapCached:       7968 kB
Active:         170048 kB
Inactive:       433160 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      2055920 kB
LowFree:       1409568 kB
SwapTotal:     4192956 kB
SwapFree:      4179744 kB
Dirty:            3460 kB
Writeback:           0 kB
Mapped:          14072 kB
Slab:            29008 kB
Committed_AS:   538624 kB
PageTables:       1436 kB
VmallocTotal: 536870911 kB
VmallocUsed:    264952 kB
VmallocChunk: 536605679 kB
HugePages_Total:     0
HugePages_Free:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 66188 MiB
End job on node67 at 2009-04-29 12:16:22