Tdc2010
Click here to load reader
-
Upload
rodrigo-huebner -
Category
Documents
-
view
846 -
download
2
Transcript of Tdc2010
![Page 1: Tdc2010](https://reader037.fdocumentos.tips/reader037/viewer/2022100519/559e84e71a28abb10b8b46ef/html5/thumbnails/1.jpg)
Programação Paralela em PythonBibliotecas e Alternativas
TDC 2010
Rodrigo Hübner
![Page 2: Tdc2010](https://reader037.fdocumentos.tips/reader037/viewer/2022100519/559e84e71a28abb10b8b46ef/html5/thumbnails/2.jpg)
Vamos discutir...
● Arquiteturas Computacionais estão se deslocando
para UCP's multicore;
● Softwares precisam explorar o paralelismo subjacente;
● Programação multithreading em Python;
● Bibliotecas existentes...
● Alternativas;
● Conclusões.
![Page 3: Tdc2010](https://reader037.fdocumentos.tips/reader037/viewer/2022100519/559e84e71a28abb10b8b46ef/html5/thumbnails/3.jpg)
Exemplo: threading
from threading import Thread
def minha_funcao(argumento): print argumento
t = Thread( target = minha_funcao, args = ('algo',))t.start()
![Page 4: Tdc2010](https://reader037.fdocumentos.tips/reader037/viewer/2022100519/559e84e71a28abb10b8b46ef/html5/thumbnails/4.jpg)
Exemplo: threading
Problema!
...Python não é totalmente thread-safe...
O que fazer?
![Page 5: Tdc2010](https://reader037.fdocumentos.tips/reader037/viewer/2022100519/559e84e71a28abb10b8b46ef/html5/thumbnails/5.jpg)
Exemplos: multiprocessing
from multiprocessing import Process
def f(nome): print 'Olá', nome
if __name__ == '__main__': p = Process(target=f, args=('Rodrigo',)) p.start() p.join()
![Page 6: Tdc2010](https://reader037.fdocumentos.tips/reader037/viewer/2022100519/559e84e71a28abb10b8b46ef/html5/thumbnails/6.jpg)
Exemplos: multiprocessing
from multiprocessing import Process, Queue
def f(q): q.put([42, None, 'hello'])
if __name__ == '__main__': q = Queue() p = Process(target=f, args=(q,)) p.start() print q.get() # prints "[42, None, 'hello']" p.join()
![Page 7: Tdc2010](https://reader037.fdocumentos.tips/reader037/viewer/2022100519/559e84e71a28abb10b8b46ef/html5/thumbnails/7.jpg)
Exemplo: Parallel Pythonimport math, sys, time, pp
def sum_primes(n): return sum([x for x in xrange(2,n) if isprime(x)])
ppservers = ()
if len(sys.argv) > 1: ncpus = int(sys.argv[1]) job_server = pp.Server(ncpus, ppservers=ppservers)else: job_server = pp.Server(ppservers=ppservers)
job1 = job_server.submit(sum_primes, (100,), (isprime,), ("math",))
result = job1()start_time = time.time()
inputs = (100000, 100100, 100200, 100300, 100400, 100500, 100600, 100700)jobs = [(input, job_server.submit(sum_primes,(input,), (isprime,), ("math",))) for input in inputs]for input, job in jobs: print "Sum of primes below", input, "is", job()
job_server.print_stats()
![Page 8: Tdc2010](https://reader037.fdocumentos.tips/reader037/viewer/2022100519/559e84e71a28abb10b8b46ef/html5/thumbnails/8.jpg)
Parallel Python
Feio!
![Page 9: Tdc2010](https://reader037.fdocumentos.tips/reader037/viewer/2022100519/559e84e71a28abb10b8b46ef/html5/thumbnails/9.jpg)
Exemplos: MPI for Python
from mpi4py import MPI
comm = MPI.COMM_WORLDrank = comm.Get_rank()
if rank == 0: data = {'a': 7, 'b': 3.14} comm.send(data, dest=1, tag=11)elif rank == 1: data = comm.recv(source=0, tag=11)
![Page 10: Tdc2010](https://reader037.fdocumentos.tips/reader037/viewer/2022100519/559e84e71a28abb10b8b46ef/html5/thumbnails/10.jpg)
Alternativas...
● Jython possui um ótimo desempenho usando threads pois não possui o GIL;
● MPI está implementada em Python.
● Criar novos processos a partir de threads em Python;
● Aproveitar padrões existentes criando novos modelos...
![Page 11: Tdc2010](https://reader037.fdocumentos.tips/reader037/viewer/2022100519/559e84e71a28abb10b8b46ef/html5/thumbnails/11.jpg)
Aplicações de modelos...
● Objetivo é aplicar modelos de programação paralela que que facilite o trabalho do programador e que forneça escalabilidade.
● Modelo de programação com fluxos de execução paralelo implícito – TIRT
![Page 12: Tdc2010](https://reader037.fdocumentos.tips/reader037/viewer/2022100519/559e84e71a28abb10b8b46ef/html5/thumbnails/12.jpg)
O módulo TIRT
● Cria trabalhadores implícitos;
● Gerenciamento a cargo do ambiente de execução;
● Balanceamento de carga (Roubo de Tarefas)
● Oferece escalabilidade.
![Page 13: Tdc2010](https://reader037.fdocumentos.tips/reader037/viewer/2022100519/559e84e71a28abb10b8b46ef/html5/thumbnails/13.jpg)
TIRT: arquitetura
![Page 14: Tdc2010](https://reader037.fdocumentos.tips/reader037/viewer/2022100519/559e84e71a28abb10b8b46ef/html5/thumbnails/14.jpg)
O módulo TIRT: Implementação
● Anotação de tarefas usando @decorators;
● multiprocessing;
● Gerenciar objetos compartilhados utilizando Managers (multiprocessing);
● NÃO utilizar pré-processamento!
![Page 15: Tdc2010](https://reader037.fdocumentos.tips/reader037/viewer/2022100519/559e84e71a28abb10b8b46ef/html5/thumbnails/15.jpg)
TIRT: Exemplos
01 from tirt import task, barrier, get_n_cpu, final, init0203 @task04 def worker(w):05 first_row = (w - 1) * HEIGHT + 106 last_row = first_row + HEIGHT - 107 barrier()08 for iters in range(1, MAX_ITERS):09 for i in range(first_row, last_row):10 # Processa pontos vermelhos11 barrier()12 for i in range(first_row, last_row):13 # Processa pontos pretos14 barrier()15 16 @final17 def validate():18 # Faz a validação dos resultados obtidos
20 @init21 def matrix_stuff(): 22 # Inicializa a matriz2324 if __name__ == "__main__":25 MAX_ITERS = 10026 N = 500027 HEIGHT = N / get_n_cpu()28 GRID = []29 matrix_stuff()30 for i in range(get_n_cpu()):31 worker(i+1)32 validate()