Trace number 3321741

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-random? (MO) 7803.3 985.721

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v35000-c147000-S419010076-063-UNKNOWN.cnf
MD5SUM07407a86c8d0f1e46c34022f68908549
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark235.041
Satisfiable
(Un)Satisfiability was proved
Number of variables35000
Number of clauses147000
Sum of the clauses size441000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3147000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.32/0.46	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.32/0.46	c INFO: solving HOME/instance-3321741-1304366546.cnf
0.32/0.46	c INFO: computing task features
0.81/0.98	c INFO: computed variable-clause graph statistics
0.81/1.00	c INFO: computed clause balance statistics
3.40/3.57	c INFO: computed variable balance statistics
5.50/5.64	c INFO: computed small-clause counts
5.50/5.65	c INFO: computed Horn-clause counts
7.10/7.21	c INFO: computed variable graph statistics
7.10/7.21	c INFO: collected features for HOME/instance-3321741-1304366546.cnf in 6.73 s
7.10/7.25	c INFO: running hybridGM3@0 for 97 with 39993 remaining (b = 0.36)
7.10/7.28	c INFO: running hybridGM3@0 for 97 with 39993 remaining (b = 0.21)
7.20/7.32	c INFO: running hybridGM3@0 for 97 with 39993 remaining (b = 0.14)
7.20/7.37	c INFO: running hybridGM3@0 for 194 with 39993 remaining (b = 0.18)
7.30/7.41	c INFO: running hybridGM3@0 for 194 with 39993 remaining (b = 0.14)
7.30/7.45	c INFO: running hybridGM3@0 for 194 with 39993 remaining (b = 0.11)
7.30/7.49	c INFO: running TNM@0 for 97 with 39993 remaining (b = 0.04)
7.40/7.53	c INFO: running hybridGM3@0 for 194 with 39992 remaining (b = 0.07)
798.55/106.58	c INFO: running hybridGM3@0 for 97 with 39893 remaining (b = 0.03)
806.48/107.58	c INFO: running hybridGM3@0 for 97 with 39892 remaining (b = 0.03)
807.28/107.63	c INFO: running hybridGM3@0 for 97 with 39892 remaining (b = 0.03)
808.04/107.72	c INFO: running hybridGM3@0 for 194 with 39892 remaining (b = 0.06)
1566.17/202.77	c INFO: running hybridGM3@99 for 97 with 39797 remaining (b = 0.03)
1574.96/203.80	c INFO: running hybridGM3@0 for 97 with 39796 remaining (b = 0.03)
1574.96/203.86	c INFO: running hybridGM3@0 for 97 with 39796 remaining (b = 0.03)
1575.74/203.92	c INFO: running hybridGM3@0 for 97 with 39796 remaining (b = 0.03)
1576.52/204.00	c INFO: running hybridGM3@98 for 97 with 39796 remaining (b = 0.03)
1590.16/205.79	c INFO: running hybridGM3@100 for 97 with 39794 remaining (b = 0.03)
1614.82/208.83	c INFO: running hybridGM3@99 for 97 with 39791 remaining (b = 0.03)
2334.06/298.99	c INFO: running hybridGM3@0 for 194 with 39701 remaining (b = 0.04)
2373.96/303.97	c INFO: running hybridGM3@295 for 97 with 39696 remaining (b = 0.02)
2374.74/304.02	c INFO: running hybridGM3@0 for 194 with 39696 remaining (b = 0.03)
2376.12/304.29	c INFO: running TNM@0 for 97 with 39696 remaining (b = 0.02)
2381.83/305.00	c INFO: running hybridGM3@94 for 97 with 39695 remaining (b = 0.02)
2382.63/305.06	c INFO: running TNM@0 for 1940 with 39695 remaining (b = 0.22)
2383.39/305.13	c INFO: running hybridGM3@0 for 194 with 39695 remaining (b = 0.03)
2384.15/305.21	c INFO: running TNM@0 for 97 with 39695 remaining (b = 0.01)
3133.66/399.19	c INFO: running hybridGM3@389 for 97 with 39601 remaining (b = 0.01)
3142.42/400.21	c INFO: running hybridGM3@0 for 194 with 39600 remaining (b = 0.02)
3175.89/404.46	c INFO: running adaptg2wsat2009++@0 for 194 with 39596 remaining (b = 0.02)
3176.64/404.54	c INFO: running hybridGM3@0 for 194 with 39595 remaining (b = 0.02)
3893.39/494.42	c INFO: running TNM@0 for 97 with 39506 remaining (b = 0.01)
3902.13/495.55	c INFO: running hybridGM3@100 for 97 with 39504 remaining (b = 0.01)
3942.78/500.62	c INFO: running TNM@0 for 97 with 39499 remaining (b = 0.01)
3951.53/501.74	c INFO: running hybridGM3@99 for 97 with 39498 remaining (b = 0.01)
4661.15/590.78	c INFO: running hybridGM3@297 for 97 with 39409 remaining (b = 0.01)
4692.16/594.67	c INFO: running hybridGM3@0 for 194 with 39405 remaining (b = 0.02)
4733.61/599.88	c INFO: running TNM@0 for 97 with 39400 remaining (b = 0.01)
4742.34/600.91	c INFO: running TNM@0 for 97 with 39399 remaining (b = 0.01)
4743.74/601.12	c INFO: running hybridGM3@94 for 97 with 39399 remaining (b = 0.01)
4756.69/602.80	c INFO: running hybridGM3@98 for 97 with 39397 remaining (b = 0.01)
4758.22/602.99	c INFO: running hybridGM3@0 for 194 with 39397 remaining (b = 0.01)
5467.66/692.06	c INFO: running hybridGM3@397 for 97 with 39308 remaining (b = 0.01)
5515.45/698.06	c INFO: running TNM@0 for 97 with 39302 remaining (b = 0.01)
5532.96/700.22	c INFO: running adaptg2wsat2009++@100 for 97 with 39300 remaining (b = 0.01)
5540.10/701.20	c INFO: running TNM@0 for 97 with 39299 remaining (b = 0.01)
5549.64/702.40	c INFO: running hybridGM3@291 for 194 with 39298 remaining (b = 0.02)
6226.45/787.33	c INFO: running TNM@0 for 97 with 39213 remaining (b = 0.01)
6258.28/791.30	c INFO: running hybridGM3@99 for 97 with 39209 remaining (b = 0.01)
6306.85/797.45	c INFO: running TNM@0 for 97 with 39203 remaining (b = 0.01)
6307.57/797.55	c INFO: running TNM@0 for 97 with 39202 remaining (b = 0.01)
6324.28/799.66	c INFO: running adaptg2wsat2009++@0 for 194 with 39200 remaining (b = 0.01)
6379.10/806.54	c INFO: running hybridGM3@101 for 97 with 39194 remaining (b = 0.01)
7016.06/886.59	c INFO: running TNM@0 for 97 with 39113 remaining (b = 0.01)
7024.81/887.68	c INFO: running TNM@0 for 97 with 39112 remaining (b = 0.01)
7105.07/897.76	c INFO: running TNM@0 for 97 with 39102 remaining (b = 0.01)
7105.82/897.88	c INFO: running TNM@0 for 97 with 39102 remaining (b = 0.01)
7121.78/899.89	c INFO: running hybridGM3@386 for 194 with 39100 remaining (b = 0.01)
7153.53/903.85	c INFO: running TNM@0 for 97 with 39096 remaining (b = 0.01)
7800.02/985.11	c INFO: running TNM@0 for 97 with 39015 remaining (b = 0.01)
7803.30/985.65	c INFO: running hybridGM3@195 for 291 with 39014 remaining (b = 0.02)
7803.30/985.65	Traceback (most recent call last):
7803.30/985.65	  File "HOME/solve", line 33, in <module>
7803.30/985.65	    plac.call(main)
7803.30/985.65	  File "/home/evaluation/evaluation/tools64/lib/python2.7/site-packages/plac-0.8.1-py2.7.egg/plac_core.py", line 311, in call
7803.30/985.69	    cmd, result = parser_from(obj).consume(arglist)
7803.30/985.69	  File "/home/evaluation/evaluation/tools64/lib/python2.7/site-packages/plac-0.8.1-py2.7.egg/plac_core.py", line 192, in consume
7803.30/985.69	    return cmd, self.func(*(args + varargs + extraopts), **kwargs)
7803.30/985.69	  File "HOME/borg/build/lib/borg/tools/solve.py", line 84, in main
7803.30/985.69	    (_, answer, _) = solver(input_path, budget, cores)
7803.30/985.69	  File "HOME/borg/build/lib/borg/bilevel.py", line 202, in __call__
7803.30/985.69	    solver.go(max_cost * borg.defaults.machine_speed)
7803.30/985.69	  File "HOME/borg/build/lib/borg/solvers.py", line 179, in go
7803.30/985.69	    self._process.start()
7803.30/985.69	  File "/home/evaluation/evaluation/tools64/lib/python2.7/multiprocessing/process.py", line 94, in start
7803.30/985.69	    assert self._popen is None, 'cannot start a process twice'
7803.30/985.69	AssertionError: cannot start a process twice

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3321741-1304366546/watcher-3321741-1304366546 -o /tmp/evaluation-result-3321741-1304366546/solver-3321741-1304366546 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3321741-1304366546.cnf -seed 1634331814 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.40 1.68 0.62 1/225 32690
/proc/meminfo: memFree=16062176/32951124 swapFree=59766108/67111528
[pid=32690] ppid=32688 vsize=15760 CPUtime=0 cores=0-7
/proc/32690/stat : 32690 (python) R 32688 32690 32572 0 -1 4202496 841 0 0 0 0 0 0 0 20 0 1 0 548255151 16138240 725 33554432000 4194304 5683644 140737373344784 18446744073709551615 4540219 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/32690/statm: 3939 725 334 364 0 426 0

[startup+0.070637 s]
/proc/loadavg: 5.40 1.68 0.62 1/225 32690
/proc/meminfo: memFree=16062176/32951124 swapFree=59766108/67111528
[pid=32690] ppid=32688 vsize=54540 CPUtime=0.04 cores=0-7
/proc/32690/stat : 32690 (python) R 32688 32690 32572 0 -1 4202496 2071 0 0 0 3 1 0 0 20 0 1 0 548255151 55848960 1828 33554432000 4194304 5683644 140737373344784 18446744073709551615 230809161573 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/32690/statm: 13635 1828 492 364 0 1285 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 54540

[startup+0.100623 s]
/proc/loadavg: 5.40 1.68 0.62 1/225 32690
/proc/meminfo: memFree=16062176/32951124 swapFree=59766108/67111528
[pid=32690] ppid=32688 vsize=104232 CPUtime=0.07 cores=0-7
/proc/32690/stat : 32690 (python) S 32688 32690 32572 0 -1 4202496 3066 0 0 0 5 2 0 0 17 0 1 0 548255151 106733568 2748 33554432000 4194304 5683644 140737373344784 18446744073709551615 230809163776 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/32690/statm: 26058 2748 847 364 0 1806 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 104232

[startup+0.3006 s]
/proc/loadavg: 5.40 1.68 0.62 1/225 32690
/proc/meminfo: memFree=16062176/32951124 swapFree=59766108/67111528
[pid=32690] ppid=32688 vsize=170756 CPUtime=0.24 cores=0-7
/proc/32690/stat : 32690 (python) S 32688 32690 32572 0 -1 4202496 6734 933 1 0 17 7 0 0 17 0 1 0 548255151 174854144 4806 33554432000 4194304 5683644 140737373344784 18446744073709551615 230809163776 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/32690/statm: 42689 4806 1336 364 0 3454 0
Current children cumulated CPU time (s) 0.24
Current children cumulated vsize (KiB) 170756

[startup+0.700509 s]
/proc/loadavg: 5.40 1.68 0.62 1/225 32690
/proc/meminfo: memFree=16062176/32951124 swapFree=59766108/67111528
[pid=32690] ppid=32688 vsize=295456 CPUtime=0.6 cores=0-7
/proc/32690/stat : 32690 (python) R 32688 32690 32572 0 -1 4202496 13476 933 1 0 51 9 0 0 17 0 1 0 548255151 302546944 11159 33554432000 4194304 5683644 140737373344784 18446744073709551615 4445115 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/32690/statm: 73897 11160 1761 364 0 9821 0
Current children cumulated CPU time (s) 0.6
Current children cumulated vsize (KiB) 295456

[startup+1.50038 s]
/proc/loadavg: 5.40 1.68 0.62 2/226 32695
/proc/meminfo: memFree=16006276/32951124 swapFree=59766108/67111528
[pid=32690] ppid=32688 vsize=312976 CPUtime=1.4 cores=0-7
/proc/32690/stat : 32690 (python) R 32688 32690 32572 0 -1 4202496 21966 933 1 0 130 10 0 0 19 0 1 0 548255151 320487424 15628 33554432000 4194304 5683644 140737373344784 18446744073709551615 4510807 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/32690/statm: 78244 15628 1826 364 0 14168 0
Current children cumulated CPU time (s) 1.4
Current children cumulated vsize (KiB) 312976

[startup+3.10107 s]
/proc/loadavg: 5.40 1.68 0.62 2/226 32695
/proc/meminfo: memFree=16005780/32951124 swapFree=59766108/67111528
[pid=32690] ppid=32688 vsize=312976 CPUtime=3 cores=0-7
/proc/32690/stat : 32690 (python) R 32688 32690 32572 0 -1 4202496 21966 933 1 0 290 10 0 0 22 0 1 0 548255151 320487424 15628 33554432000 4194304 5683644 140737373344784 18446744073709551615 230808856978 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/32690/statm: 78244 15628 1826 364 0 14168 0
Current children cumulated CPU time (s) 3
Current children cumulated vsize (KiB) 312976

[startup+6.30048 s]
/proc/loadavg: 5.13 1.69 0.63 2/226 32695
/proc/meminfo: memFree=16005588/32951124 swapFree=59766108/67111528
[pid=32690] ppid=32688 vsize=312976 CPUtime=6.2 cores=0-7
/proc/32690/stat : 32690 (python) R 32688 32690 32572 0 -1 4202496 21973 933 1 0 610 10 0 0 25 0 1 0 548255151 320487424 15635 33554432000 4194304 5683644 140737373344784 18446744073709551615 4603456 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/32690/statm: 78244 15635 1833 364 0 14168 0
Current children cumulated CPU time (s) 6.2
Current children cumulated vsize (KiB) 312976

[startup+12.7013 s]
/proc/loadavg: 5.36 1.79 0.67 9/250 32719
/proc/meminfo: memFree=15728872/32951124 swapFree=59766108/67111528
[pid=32690] ppid=32688 vsize=392304 CPUtime=7.43 cores=0-7
/proc/32690/stat : 32690 (python) S 32688 32690 32572 0 -1 4202496 28845 933 1 0 730 13 0 0 24 0 9 0 548255151 401719296 15040 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820992459 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/32690/statm: 98076 15040 1889 364 0 33973 0
[pid=32696] ppid=32690 vsize=310268 CPUtime=0 cores=0-7
/proc/32696/stat : 32696 (python) S 32690 32690 32572 0 -1 4202560 907 0 0 0 0 0 0 0 15 0 1 0 548255876 317714432 13495 33554432000 4194304 5683644 140737373344784 18446744073709551615 230809185807 0 0 16781312 514 4091723125656355073 0 0 17 7 0 0 0
/proc/32696/statm: 77567 13495 376 364 0 13485 0
[pid=32698] ppid=32696 vsize=268424 CPUtime=5.42 cores=0-7
/proc/32698/stat : 32698 (hybridGM3) R 32696 32698 32698 0 -1 4202496 8084 0 0 0 541 1 0 0 25 0 1 0 548255876 274866176 7821 33554432000 134512640 135204385 4290456304 18446744073709551615 134546099 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/32698/statm: 67106 7821 64 169 0 66936 0
[pid=32699] ppid=32690 vsize=320524 CPUtime=0 cores=0-7
/proc/32699/stat : 32699 (python) S 32690 32690 32572 0 -1 4202560 908 0 0 0 0 0 0 0 15 0 1 0 548255880 328216576 13493 33554432000 4194304 5683644 140737373344784 18446744073709551615 230809185862 0 0 16781312 514 4091726973947052290 0 0 17 6 0 0 0
/proc/32699/statm: 80131 13493 370 364 0 16046 0
[pid=32701] ppid=32699 vsize=268424 CPUtime=5.4 cores=0-7
/proc/32701/stat : 32701 (hybridGM3) R 32699 32701 32701 0 -1 4202496 8085 0 0 0 539 1 0 0 25 0 1 0 548255880 274866176 7824 33554432000 134512640 135204385 4288067024 18446744073709551615 134546059 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/32701/statm: 67106 7824 64 169 0 66936 0
[pid=32702] ppid=32690 vsize=330780 CPUtime=0 cores=0-7
/proc/32702/stat : 32702 (python) S 32690 32690 32572 0 -1 4202560 908 0 0 0 0 0 0 0 15 0 1 0 548255883 338718720 13497 33554432000 4194304 5683644 140737373344784 18446744073709551615 230809185862 0 0 16781312 514 10141424393046163716 0 0 17 4 0 0 0
/proc/32702/statm: 82695 13497 370 364 0 18607 0
[pid=32704] ppid=32702 vsize=268424 CPUtime=5.3 cores=0-7
/proc/32704/stat : 32704 (hybridGM3) R 32702 32704 32704 0 -1 4202496 8088 0 0 0 529 1 0 0 25 0 1 0 548255884 274866176 7827 33554432000 134512640 135204385 4291405936 18446744073709551615 134546082 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/32704/statm: 67106 7827 64 169 0 66936 0
[pid=32705] ppid=32690 vsize=341036 CPUtime=0 cores=0-7
/proc/32705/stat : 32705 (python) S 32690 32690 32572 0 -1 4202560 940 0 0 0 0 0 0 0 15 0 1 0 548255888 349220864 13501 33554432000 4194304 5683644 140737373344784 18446744073709551615 230809185862 0 0 16781312 514 4146680015347482887 0 0 17 0 0 0 0
/proc/32705/statm: 85259 13501 370 364 0 21168 0
[pid=32707] ppid=32705 vsize=268420 CPUtime=5.31 cores=0-7
/proc/32707/stat : 32707 (hybridGM3) R 32705 32707 32707 0 -1 4202496 8080 0 0 0 531 0 0 0 25 0 1 0 548255888 274862080 7818 33554432000 134512640 135204385 4294570816 18446744073709551615 134537403 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/32707/statm: 67105 7818 64 169 0 66935 0
[pid=32708] ppid=32690 vsize=351292 CPUtime=0 cores=0-7
/proc/32708/stat : 32708 (python) S 32690 32690 32572 0 -1 4202560 952 0 0 0 0 0 0 0 15 0 1 0 548255892 359723008 13505 33554432000 4194304 5683644 140737373344784 18446744073709551615 230809185862 0 0 16781312 514 4052347690120347905 0 0 17 2 0 0 0
/proc/32708/statm: 87823 13505 370 364 0 23729 0
[pid=32710] ppid=32708 vsize=268424 CPUtime=5.26 cores=0-7
/proc/32710/stat : 32710 (hybridGM3) R 32708 32710 32710 0 -1 4202496 8084 0 0 0 526 0 0 0 25 0 1 0 548255892 274866176 7821 33554432000 134512640 135204385 4291482128 18446744073709551615 134545837 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/32710/statm: 67106 7821 64 169 0 66936 0
[pid=32711] ppid=32690 vsize=361548 CPUtime=0 cores=0-7
/proc/32711/stat : 32711 (python) S 32690 32690 32572 0 -1 4202560 949 0 0 0 0 0 0 0 15 0 1 0 548255896 370225152 13508 33554432000 4194304 5683644 140737373344784 18446744073709551615 230809185862 0 0 16781312 514 1436325853641277698 0 0 17 6 0 0 0
/proc/32711/statm: 90387 13508 369 364 0 26290 0
[pid=32713] ppid=32711 vsize=268420 CPUtime=5.2 cores=0-7
/proc/32713/stat : 32713 (hybridGM3) R 32711 32713 32713 0 -1 4202496 8092 0 0 0 519 1 0 0 25 0 1 0 548255896 274862080 7827 33554432000 134512640 135204385 4291354624 18446744073709551615 134537357 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/32713/statm: 67105 7827 64 169 0 66935 0
[pid=32714] ppid=32690 vsize=371804 CPUtime=0 cores=0-7
/proc/32714/stat : 32714 (python) S 32690 32690 32572 0 -1 4202560 963 0 0 0 0 0 0 0 18 0 1 0 548255900 380727296 13513 33554432000 4194304 5683644 140737373344784 18446744073709551615 230809185862 0 0 16781312 514 4190590111714345218 0 0 17 0 0 0 0
/proc/32714/statm: 92951 13513 370 364 0 28851 0
[pid=32716] ppid=32714 vsize=465044 CPUtime=5.14 cores=0-7
/proc/32716/stat : 32716 (TNM) R 32714 32716 32716 0 -1 4202496 6013 0 0 0 514 0 0 0 25 0 1 0 548255900 476205056 5749 33554432000 134512640 135008748 4286688176 18446744073709551615 134524720 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/32716/statm: 116261 5749 46 122 0 115161 0
[pid=32717] ppid=32690 vsize=382060 CPUtime=0.01 cores=0-7
/proc/32717/stat : 32717 (python) S 32690 32690 32572 0 -1 4202560 1011 0 0 0 0 1 0 0 18 0 1 0 548255904 391229440 13523 33554432000 4194304 5683644 140737373344784 18446744073709551615 230809185862 0 0 16781312 514 4058429913567297793 0 0 17 6 0 0 0
/proc/32717/statm: 95515 13523 376 364 0 31412 0
[pid=32719] ppid=32717 vsize=268420 CPUtime=5.12 cores=0-7
/proc/32719/stat : 32719 (hybridGM3) R 32717 32719 32719 0 -1 4202496 8089 0 0 0 511 1 0 0 25 0 1 0 548255905 274862080 7820 33554432000 134512640 135204385 4294929424 18446744073709551615 134536806 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/32719/statm: 67105 7820 64 169 0 66935 0
[pid=32690/tid=32697] ppid=32688 vsize=392304 CPUtime=0 cores=0-7
/proc/32690/task/32697/stat : 32697 (python) S 32688 32690 32572 0 -1 4202560 27 933 0 0 0 0 0 0 25 0 9 0 548255876 401719296 15040 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 6 0 0 0
[pid=32690/tid=32700] ppid=32688 vsize=392304 CPUtime=0 cores=0-7
/proc/32690/task/32700/stat : 32700 (python) S 32688 32690 32572 0 -1 4202560 22 933 0 0 0 0 0 0 25 0 9 0 548255880 401719296 15040 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 6 0 0 0
[pid=32690/tid=32703] ppid=32688 vsize=392304 CPUtime=0 cores=0-7
/proc/32690/task/32703/stat : 32703 (python) S 32688 32690 32572 0 -1 4202560 23 933 0 0 0 0 0 0 20 0 9 0 548255884 401719296 15040 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 4 0 0 0
[pid=32690/tid=32706] ppid=32688 vsize=392304 CPUtime=0 cores=0-7
/proc/32690/task/32706/stat : 32706 (python) S 32688 32690 32572 0 -1 4202560 26 933 0 0 0 0 0 0 22 0 9 0 548255888 401719296 15040 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 0 0 0 0
[pid=32690/tid=32709] ppid=32688 vsize=392304 CPUtime=0 cores=0-7
/proc/32690/task/32709/stat : 32709 (python) S 32688 32690 32572 0 -1 4202560 27 933 0 0 0 0 0 0 20 0 9 0 548255892 401719296 15040 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 2 0 0 0
[pid=32690/tid=32712] ppid=32688 vsize=392304 CPUtime=0 cores=0-7
/proc/32690/task/32712/stat : 32712 (python) S 32688 32690 32572 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 9 0 548255896 401719296 15040 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 6 0 0 0
[pid=32690/tid=32715] ppid=32688 vsize=392304 CPUtime=0 cores=0-7
/proc/32690/task/32715/stat : 32715 (python) S 32688 32690 32572 0 -1 4202560 30 933 0 0 0 0 0 0 22 0 9 0 548255900 401719296 15040 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 0 0 0 0
[pid=32690/tid=32718] ppid=32688 vsize=392304 CPUtime=0 cores=0-7
/proc/32690/task/32718/stat : 32718 (python) S 32688 32690 32572 0 -1 4202560 32 933 0 0 0 0 0 0 24 0 9 0 548255905 401719296 15040 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 49.59
Current children cumulated vsize (KiB) 5505616

[startup+25.5019 s]
/proc/loadavg: 5.94 2.10 0.79 9/250 32719
/proc/meminfo: memFree=15730812/32951124 swapFree=59766108/67111528
[pid=32690] ppid=32688 vsize=392304 CPUtime=7.43 cores=0-7
/proc/32690/stat : 32690 (python) S 32688 32690 32572 0 -1 4202496 28845 933 1 0 730 13 0 0 24 0 9 0 548255151 401719296 15040 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820992459 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/32690/statm: 98076 15040 1889 364 0 33973 0
[pid=32696] ppid=32690 vsize=310268 CPUtime=0.02 cores=0-7
/proc/32696/stat : 32696 (python) S 32690 32690 32572 0 -1 4202560 914 0 0 0 1 1 0 0 15 0 1 0 548255876 317714432 13495 33554432000 4194304 5683644 140737373344784 18446744073709551615 230809185807 0 0 16781312 514 3424002354196742401 0 0 17 7 0 0 0
/proc/32696/statm: 77567 13495 376 364 0 13485 0
[pid=32698] ppid=32696 vsize=268424 CPUtime=18.21 cores=0-7
/proc/32698/stat : 32698 (hybridGM3) R 32696 32698 32698 0 -1 4202496 8089 0 0 0 1820 1 0 0 25 0 1 0 548255876 274866176 7826 33554432000 134512640 135204385 4290456304 18446744073709551615 134536737 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/32698/statm: 67106 7826 64 169 0 66936 0
[pid=32699] ppid=32690 vsize=320524 CPUtime=0.02 cores=0-7
/proc/32699/stat : 32699 (python) S 32690 32690 32572 0 -1 4202560 913 0 0 0 1 1 0 0 15 0 1 0 548255880 328216576 13493 33554432000 4194304 5683644 140737373344784 18446744073709551615 230809185862 0 0 16781312 514 3424002354196742402 0 0 17 6 0 0 0
/proc/32699/statm: 80131 13493 370 364 0 16046 0
[pid=32701] ppid=32699 vsize=268424 CPUtime=18.2 cores=0-7
/proc/32701/stat : 32701 (hybridGM3) R 32699 32701 32701 0 -1 4202496 8085 0 0 0 1819 1 0 0 25 0 1 0 548255880 274866176 7824 33554432000 134512640 135204385 4288067024 18446744073709551615 134537573 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/32701/statm: 67106 7824 64 169 0 66936 0
[pid=32702] ppid=32690 vsize=330780 CPUtime=0.02 cores=0-7
/proc/32702/stat : 32702 (python) S 32690 32690 32572 0 -1 4202560 911 0 0 0 1 1 0 0 15 0 1 0 548255883 338718720 13497 33554432000 4194304 5683644 140737373344784 18446744073709551615 230809185862 0 0 16781312 514 3424002354196742404 0 0 17 4 0 0 0
/proc/32702/statm: 82695 13497 370 364 0 18607 0
[pid=32704] ppid=32702 vsize=268424 CPUtime=17.95 cores=0-7
/proc/32704/stat : 32704 (hybridGM3) R 32702 32704 32704 0 -1 4202496 8088 0 0 0 1794 1 0 0 25 0 1 0 548255884 274866176 7827 33554432000 134512640 135204385 4291405936 18446744073709551615 134537292 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/32704/statm: 67106 7827 64 169 0 66936 0
[pid=32705] ppid=32690 vsize=341036 CPUtime=0.02 cores=0-7
/proc/32705/stat : 32705 (python) S 32690 32690 32572 0 -1 4202560 948 0 0 0 1 1 0 0 15 0 1 0 548255888 349220864 13501 33554432000 4194304 5683644 140737373344784 18446744073709551615 230809185862 0 0 16781312 514 3424002354196742407 0 0 17 0 0 0 0
/proc/32705/statm: 85259 13501 370 364 0 21168 0
[pid=32707] ppid=32705 vsize=268420 CPUtime=18.11 cores=0-7
/proc/32707/stat : 32707 (hybridGM3) R 32705 32707 32707 0 -1 4202496 8082 0 0 0 1811 0 0 0 25 0 1 0 548255888 274862080 7820 33554432000 134512640 135204385 4294570816 18446744073709551615 134546059 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/32707/statm: 67105 7820 64 169 0 66935 0
[pid=32708] ppid=32690 vsize=351292 CPUtime=0.02 cores=0-7
/proc/32708/stat : 32708 (python) S 32690 32690 32572 0 -1 4202560 955 0 0 0 1 1 0 0 15 0 1 0 548255892 359723008 13505 33554432000 4194304 5683644 140737373344784 18446744073709551615 230809185862 0 0 16781312 514 3424002354196742401 0 0 17 2 0 0 0
/proc/32708/statm: 87823 13505 370 364 0 23729 0
[pid=32710] ppid=32708 vsize=268424 CPUtime=18.04 cores=0-7
/proc/32710/stat : 32710 (hybridGM3) R 32708 32710 32710 0 -1 4202496 8086 0 0 0 1804 0 0 0 25 0 1 0 548255892 274866176 7823 33554432000 134512640 135204385 4291482128 18446744073709551615 134537403 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/32710/statm: 67106 7823 64 169 0 66936 0

################
# More data... #
################

/proc/32713/stat : 32713 (hybridGM3) T 32711 32713 32713 0 -1 4202496 8092 0 0 0 29691 1 0 0 25 0 1 0 548255896 274862080 7827 33554432000 134512640 135204385 4291354624 18446744073709551615 134547676 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/32713/statm: 67105 7827 64 169 0 66935 0
[pid=32711/tid=32737] ppid=32690 vsize=361548 CPUtime=0 cores=0-7
/proc/32711/task/32737/stat : 32737 (python) S 32690 32690 32572 0 -1 4202560 52 0 0 0 0 0 0 0 15 0 2 0 548275539 370225152 13539 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 514 18446604448952146624 0 0 -1 5 0 0 0
[pid=32714] ppid=32690 vsize=371804 CPUtime=1.53 cores=0-7
/proc/32714/stat : 32714 (python) S 32690 32690 32572 0 -1 4202560 1237 0 0 0 76 77 0 0 18 0 2 0 548255900 380727296 13549 33554432000 4194304 5683644 140737373344784 18446744073709551615 230809185862 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/32714/statm: 92951 13549 405 364 0 28851 0
[pid=32716] ppid=32714 vsize=465044 CPUtime=778.86 cores=0-7
/proc/32716/stat : 32716 (TNM) R 32714 32716 32716 0 -1 4202496 6013 0 0 0 77884 2 0 0 25 0 1 0 548255900 476205056 5749 33554432000 134512640 135008748 4286688176 18446744073709551615 134529183 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/32716/statm: 116261 5749 46 122 0 115161 0
[pid=32714/tid=32728] ppid=32690 vsize=371804 CPUtime=0 cores=0-7
/proc/32714/task/32728/stat : 32728 (python) S 32690 32690 32572 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 548265919 380727296 13549 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 514 18446604453888391040 0 0 -1 0 0 0 0
[pid=32717] ppid=32690 vsize=382060 CPUtime=0.5 cores=0-7
/proc/32717/stat : 32717 (python) S 32690 32690 32572 0 -1 4202560 1236 0 0 0 28 22 0 0 18 0 2 0 548255904 391229440 13553 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820992459 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/32717/statm: 95515 13553 405 364 0 31412 0
[pid=32719] ppid=32717 vsize=268420 CPUtime=295.67 cores=0-7
/proc/32719/stat : 32719 (hybridGM3) T 32717 32719 32719 0 -1 4202496 8095 0 0 0 29566 1 0 0 25 0 1 0 548255905 274862080 7826 33554432000 134512640 135204385 4294929424 18446744073709551615 134536791 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/32719/statm: 67105 7826 64 169 0 66935 0
[pid=32717/tid=32738] ppid=32690 vsize=382060 CPUtime=0 cores=0-7
/proc/32717/task/32738/stat : 32738 (python) S 32690 32690 32572 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 548275547 391229440 13553 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 514 18446604453857620096 0 0 -1 6 0 0 0
[pid=32729] ppid=32690 vsize=392316 CPUtime=0.5 cores=0-7
/proc/32729/stat : 32729 (python) S 32690 32690 32572 0 -1 4202560 1205 0 0 0 25 25 0 0 15 0 2 0 548265923 401731584 13551 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820992459 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/32729/statm: 98079 13551 399 364 0 33973 0
[pid=32731] ppid=32729 vsize=268424 CPUtime=291.25 cores=0-7
/proc/32731/stat : 32731 (hybridGM3) T 32729 32731 32731 0 -1 4202496 8092 0 0 0 29122 3 0 0 25 0 1 0 548265925 274866176 7824 33554432000 134512640 135204385 4287804288 18446744073709551615 134536922 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/32731/statm: 67106 7824 64 169 0 66936 0
[pid=32729/tid=32748] ppid=32690 vsize=392316 CPUtime=0 cores=0-7
/proc/32729/task/32748/stat : 32748 (python) S 32690 32690 32572 0 -1 4202560 47 0 0 0 0 0 0 0 15 0 2 0 548285575 401731584 13551 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 514 18446604451036439552 0 0 -1 6 0 0 0
[pid=32742] ppid=32690 vsize=468108 CPUtime=0.94 cores=0-7
/proc/32742/stat : 32742 (python) S 32690 32690 32572 0 -1 4202560 1204 0 0 0 43 51 0 0 15 0 2 0 548285050 479342592 13600 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820992459 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/32742/statm: 117027 13600 399 364 0 52918 0
[pid=32744] ppid=32742 vsize=268420 CPUtime=487.84 cores=0-7
/proc/32744/stat : 32744 (hybridGM3) T 32742 32744 32744 0 -1 4202496 8092 0 0 0 48783 1 0 0 25 0 1 0 548285051 274862080 7821 33554432000 134512640 135204385 4286839808 18446744073709551615 134537294 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/32744/statm: 67105 7821 64 169 0 66935 0
[pid=32742/tid=309] ppid=32690 vsize=468108 CPUtime=0 cores=0-7
/proc/32742/task/309/stat : 309 (python) S 32690 32690 32572 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 548304698 479342592 13600 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 514 18446604448377187200 0 0 -1 2 0 0 0
[pid=32745] ppid=32690 vsize=478364 CPUtime=0.36 cores=0-7
/proc/32745/stat : 32745 (python) S 32690 32690 32572 0 -1 4202560 1217 0 0 0 20 16 0 0 15 0 2 0 548285553 489844736 13618 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820992459 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/32745/statm: 119591 13618 399 364 0 55479 0
[pid=32747] ppid=32745 vsize=268424 CPUtime=196.29 cores=0-7
/proc/32747/stat : 32747 (hybridGM3) T 32745 32747 32747 0 -1 4202496 8098 0 0 0 19627 2 0 0 25 0 1 0 548285553 274866176 7828 33554432000 134512640 135204385 4291304800 18446744073709551615 134537292 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/32747/statm: 67106 7828 64 169 0 66936 0
[pid=32745/tid=310] ppid=32690 vsize=478364 CPUtime=0 cores=0-7
/proc/32745/task/310/stat : 310 (python) S 32690 32690 32572 0 -1 4202560 47 0 0 0 0 0 0 0 16 0 2 0 548305204 489844736 13618 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 514 18446604448377184512 0 0 -1 6 0 0 0
[pid=32749] ppid=32690 vsize=488620 CPUtime=0.18 cores=0-7
/proc/32749/stat : 32749 (python) S 32690 32690 32572 0 -1 4202560 1143 0 0 0 9 9 0 0 19 0 2 0 548285580 500346880 13622 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820992459 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/32749/statm: 122155 13622 399 364 0 58040 0
[pid=32751] ppid=32749 vsize=465044 CPUtime=100.17 cores=0-7
/proc/32751/stat : 32751 (TNM) T 32749 32751 32751 0 -1 4202496 6021 0 0 0 10016 1 0 0 25 0 1 0 548285581 476205056 5749 33554432000 134512640 135008748 4287738944 18446744073709551615 134527475 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/32751/statm: 116261 5749 46 122 0 115161 0
[pid=32749/tid=32766] ppid=32690 vsize=488620 CPUtime=0 cores=0-7
/proc/32749/task/32766/stat : 32766 (python) S 32690 32690 32572 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 548295599 500346880 13622 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 514 18446604449090262080 0 0 -1 4 0 0 0
[pid=32752] ppid=32690 vsize=498876 CPUtime=0.54 cores=0-7
/proc/32752/stat : 32752 (python) S 32690 32690 32572 0 -1 4202560 1202 0 0 0 30 24 0 0 16 0 2 0 548285664 510849024 13626 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820992459 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/32752/statm: 124719 13626 399 364 0 60601 0
[pid=32754] ppid=32752 vsize=268420 CPUtime=295.02 cores=0-7
/proc/32754/stat : 32754 (hybridGM3) T 32752 32754 32754 0 -1 4202496 8095 0 0 0 29501 1 0 0 25 0 1 0 548285664 274862080 7826 33554432000 134512640 135204385 4290204928 18446744073709551615 134537123 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/32754/statm: 67105 7826 64 169 0 66935 0
[pid=32752/tid=323] ppid=32690 vsize=498876 CPUtime=0 cores=0-7
/proc/32752/task/323/stat : 323 (python) S 32690 32690 32572 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 548305317 510849024 13626 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 514 18446604450948059328 0 0 -1 3 0 0 0
[pid=32755] ppid=32690 vsize=509132 CPUtime=0.18 cores=0-7
/proc/32755/stat : 32755 (python) S 32690 32690 32572 0 -1 4202560 1200 0 0 0 9 9 0 0 19 0 2 0 548285673 521351168 13630 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820992459 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/32755/statm: 127283 13630 399 364 0 63162 0
[pid=32757] ppid=32755 vsize=465044 CPUtime=98.13 cores=0-7
/proc/32757/stat : 32757 (TNM) T 32755 32757 32757 0 -1 4202496 6024 0 0 0 9812 1 0 0 25 0 1 0 548285673 476205056 5749 33554432000 134512640 135008748 4287102656 18446744073709551615 134529448 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/32757/statm: 116261 5749 46 122 0 115161 0
[pid=32755/tid=32763] ppid=32690 vsize=509132 CPUtime=0 cores=0-7
/proc/32755/task/32763/stat : 32763 (python) S 32690 32690 32572 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 548295591 521351168 13630 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 514 18446604451036438656 0 0 -1 3 0 0 0
[pid=32760] ppid=32690 vsize=519388 CPUtime=0.57 cores=0-7
/proc/32760/stat : 32760 (python) S 32690 32690 32572 0 -1 4202560 1294 0 0 0 29 28 0 0 15 0 2 0 548295173 531853312 13642 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820992459 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/32760/statm: 129847 13642 407 364 0 65723 0
[pid=32762] ppid=32760 vsize=268428 CPUtime=292.16 cores=0-7
/proc/32762/stat : 32762 (hybridGM3) T 32760 32762 32762 0 -1 4202496 8109 0 0 0 29214 2 0 0 25 0 1 0 548295173 274870272 7831 33554432000 134512640 135204385 4293155296 18446744073709551615 134537292 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/32762/statm: 67107 7831 64 169 0 66937 0
[pid=32760/tid=340] ppid=32690 vsize=519388 CPUtime=0 cores=0-7
/proc/32760/task/340/stat : 340 (python) S 32690 32690 32572 0 -1 4202560 49 0 0 0 0 0 0 0 15 0 2 0 548315423 531853312 13642 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 514 18446604453526833024 0 0 -1 7 0 0 0
[pid=32764] ppid=32690 vsize=529644 CPUtime=0.57 cores=0-7
/proc/32764/stat : 32764 (python) S 32690 32690 32572 0 -1 4202560 1255 0 0 0 31 26 0 0 18 0 2 0 548295597 542355456 13799 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820992459 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/32764/statm: 132411 13799 406 364 0 68284 0
[pid=32767] ppid=32764 vsize=431704 CPUtime=293.53 cores=0-7
/proc/32767/stat : 32767 (adaptg2wsat2009) T 32764 32767 32767 0 -1 4202496 6250 0 0 0 29352 1 0 0 25 0 1 0 548295599 442064896 5982 33554432000 134512640 135107411 4292906496 18446744073709551615 134531537 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/32767/statm: 107926 5982 59 146 0 106802 0
[pid=32764/tid=335] ppid=32690 vsize=529644 CPUtime=0 cores=0-7
/proc/32764/task/335/stat : 335 (python) S 32690 32690 32572 0 -1 4202560 177 0 0 0 0 0 0 0 18 0 2 0 548315233 542355456 13799 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 514 18446604453526832128 0 0 -1 0 0 0 0
[pid=32690/tid=32697] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/32697/stat : 32697 (python) S 32688 32690 32572 0 -1 4202560 35 933 0 0 0 0 0 0 18 0 36 0 548255876 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 6 0 0 0
[pid=32690/tid=32700] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/32700/stat : 32700 (python) S 32688 32690 32572 0 -1 4202560 39 933 0 0 0 0 0 0 18 0 36 0 548255880 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 6 0 0 0
[pid=32690/tid=32703] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/32703/stat : 32703 (python) S 32688 32690 32572 0 -1 4202560 29 933 0 0 0 0 0 0 18 0 36 0 548255884 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 4 0 0 0
[pid=32690/tid=32706] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/32706/stat : 32706 (python) S 32688 32690 32572 0 -1 4202560 31 933 0 0 0 0 0 0 18 0 36 0 548255888 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 0 0 0 0
[pid=32690/tid=32709] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/32709/stat : 32709 (python) S 32688 32690 32572 0 -1 4202560 30 933 0 0 0 0 0 0 18 0 36 0 548255892 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 2 0 0 0
[pid=32690/tid=32712] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/32712/stat : 32712 (python) S 32688 32690 32572 0 -1 4202560 34 933 0 0 0 0 0 0 18 0 36 0 548255896 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 6 0 0 0
[pid=32690/tid=32715] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/32715/stat : 32715 (python) S 32688 32690 32572 0 -1 4202560 36 933 0 0 0 0 0 0 18 0 36 0 548255900 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 6 0 0 0
[pid=32690/tid=32718] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/32718/stat : 32718 (python) S 32688 32690 32572 0 -1 4202560 35 933 0 0 0 0 0 0 18 0 36 0 548255905 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 6 0 0 0
[pid=32690/tid=32730] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/32730/stat : 32730 (python) S 32688 32690 32572 0 -1 4202560 32 933 0 0 0 0 0 0 18 0 36 0 548265923 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 1 0 0 0
[pid=32690/tid=32743] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/32743/stat : 32743 (python) S 32688 32690 32572 0 -1 4202560 47 933 0 0 0 0 0 0 18 0 36 0 548285050 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 6 0 0 0
[pid=32690/tid=32746] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/32746/stat : 32746 (python) S 32688 32690 32572 0 -1 4202560 32 933 0 0 0 0 0 0 20 0 36 0 548285553 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 6 0 0 0
[pid=32690/tid=32750] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/32750/stat : 32750 (python) S 32688 32690 32572 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 36 0 548285580 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 6 0 0 0
[pid=32690/tid=32753] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/32753/stat : 32753 (python) S 32688 32690 32572 0 -1 4202560 39 933 0 0 0 0 0 0 18 0 36 0 548285664 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 6 0 0 0
[pid=32690/tid=32756] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/32756/stat : 32756 (python) S 32688 32690 32572 0 -1 4202560 28 933 0 0 0 0 0 0 21 0 36 0 548285673 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 6 0 0 0
[pid=32690/tid=32761] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/32761/stat : 32761 (python) S 32688 32690 32572 0 -1 4202560 39 933 0 0 0 0 0 0 18 0 36 0 548295173 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 1 0 0 0
[pid=32690/tid=32765] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/32765/stat : 32765 (python) S 32688 32690 32572 0 -1 4202560 123 933 0 0 0 0 0 0 18 0 36 0 548295597 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 4 0 0 0
[pid=32690/tid=301] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/301/stat : 301 (python) S 32688 32690 32572 0 -1 4202560 43 933 0 0 0 0 0 0 18 0 36 0 548295606 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 6 0 0 0
[pid=32690/tid=307] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/307/stat : 307 (python) S 32688 32690 32572 0 -1 4202560 31 933 0 0 0 0 0 0 20 0 36 0 548304593 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 1 0 0 0
[pid=32690/tid=321] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/321/stat : 321 (python) S 32688 32690 32572 0 -1 4202560 39 933 0 0 0 0 0 0 20 0 36 0 548305213 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 6 0 0 0
[pid=32690/tid=328] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/328/stat : 328 (python) S 32688 32690 32572 0 -1 4202560 46 933 0 0 0 0 0 0 18 0 36 0 548314619 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 6 0 0 0
[pid=32690/tid=333] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/333/stat : 333 (python) S 32688 32690 32572 0 -1 4202560 40 933 0 0 0 0 0 0 20 0 36 0 548315140 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 6 0 0 0
[pid=32690/tid=337] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/337/stat : 337 (python) S 32688 32690 32572 0 -1 4202560 36 933 0 0 0 0 0 0 19 0 36 0 548315242 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 6 0 0 0
[pid=32690/tid=342] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/342/stat : 342 (python) S 32688 32690 32572 0 -1 4202560 52 933 0 0 0 0 0 0 18 0 36 0 548315451 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 6 0 0 0
[pid=32690/tid=347] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/347/stat : 347 (python) S 32688 32690 32572 0 -1 4202560 33 933 0 0 0 0 0 0 20 0 36 0 548324958 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 1 0 0 0
[pid=32690/tid=352] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/352/stat : 352 (python) S 32688 32690 32572 0 -1 4202560 29 933 0 0 0 0 0 0 20 0 36 0 548325272 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 6 0 0 0
[pid=32690/tid=358] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/358/stat : 358 (python) S 32688 32690 32572 0 -1 4202560 30 933 0 0 0 0 0 0 19 0 36 0 548333884 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 6 0 0 0
[pid=32690/tid=363] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/363/stat : 363 (python) S 32688 32690 32572 0 -1 4202560 24 933 0 0 0 0 0 0 20 0 36 0 548334896 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 6 0 0 0
[pid=32690/tid=366] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/366/stat : 366 (python) S 32688 32690 32572 0 -1 4202560 25 933 0 0 0 0 0 0 22 0 36 0 548334907 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 0 0 0 0
[pid=32690/tid=370] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/370/stat : 370 (python) S 32688 32690 32572 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 36 0 548335117 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 0 0 0 0
[pid=32690/tid=376] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/376/stat : 376 (python) S 32688 32690 32572 0 -1 4202560 29 933 0 0 0 0 0 0 20 0 36 0 548343810 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 1 0 0 0
[pid=32690/tid=380] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/380/stat : 380 (python) S 32688 32690 32572 0 -1 4202560 199 933 0 0 0 0 0 0 19 0 36 0 548343919 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 0 0 0 0
[pid=32690/tid=385] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/385/stat : 385 (python) S 32688 32690 32572 0 -1 4202560 33 933 0 0 0 0 0 0 19 0 36 0 548344927 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 0 0 0 0
[pid=32690/tid=389] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/389/stat : 389 (python) S 32688 32690 32572 0 -1 4202560 36 933 0 0 0 0 0 0 21 0 36 0 548344939 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 1 0 0 0
[pid=32690/tid=392] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/392/stat : 392 (python) S 32688 32690 32572 0 -1 4202560 29 933 0 0 0 0 0 0 20 0 36 0 548345536 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 0 0 0 0
[pid=32690/tid=398] ppid=32688 vsize=734752 CPUtime=0 cores=0-7
/proc/32690/task/398/stat : 398 (python) S 32688 32690 32572 0 -1 4202560 27 933 0 0 0 0 0 0 19 0 36 0 548353662 752386048 15568 33554432000 4194304 5683644 140737373344784 18446744073709551615 230820989361 0 0 16781312 2 18446604451036441344 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 7803.3
Current children cumulated vsize (KiB) 32733016

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 32690 and gives
#  childrusage.ru_utime.tv_sec=1640
#  childrusage.ru_utime.tv_usec=45675
#  childrusage.ru_stime.tv_sec=7
#  childrusage.ru_stime.tv_usec=885801
# CPU time returned by wait4() is 1647.93
# while last known CPU time is 7803.3
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 985.721
CPU time (s): 7803.3
CPU user time (s): 7795.4
CPU system time (s): 7.9
CPU usage (%): 791.633
Max. virtual memory (cumulated for all children) (KiB): 32733016

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1640.05
system time used= 7.8858
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 150860
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 10793
involuntary context switches= 7831

runsolver used 6.94794 second user time and 20.1439 second system time

The end

Launcher Data 

Begin job on node104 at 2011-05-02 22:02:27
IDJOB=3321741
IDBENCH=82519
IDSOLVER=1712
FILE ID=node104/3321741-1304366546
RUNJOBID= node104-1304366469-32586
PBS_JOBID= 13181766
Free space on /tmp= 73400 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v35000-c147000-S419010076-063-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3321741-1304366546/watcher-3321741-1304366546 -o /tmp/evaluation-result-3321741-1304366546/solver-3321741-1304366546 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3321741-1304366546.cnf -seed 1634331814 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 07407a86c8d0f1e46c34022f68908549
RANDOM SEED=1634331814

node104.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5293.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.855
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      16062332 kB
Buffers:         18760 kB
Cached:         299508 kB
SwapCached:      43920 kB
Active:        9058244 kB
Inactive:      7646016 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      16062332 kB
SwapTotal:    67111528 kB
SwapFree:     59766108 kB
Dirty:           34988 kB
Writeback:           0 kB
AnonPages:    16353384 kB
Mapped:          16908 kB
Slab:            61600 kB
PageTables:      58312 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 26906536 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73368 MiB
End job on node104 at 2011-05-02 22:18:55