Pré-processamento em Big Data
-
Upload
joao-gabriel-lima -
Category
Technology
-
view
235 -
download
0
description
Transcript of Pré-processamento em Big Data
![Page 2: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/2.jpg)
![Page 3: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/3.jpg)
Importância do Pré-Processamento
● Seleção de atributos
● Limpeza dos Dados
● Transformação
● Construção de atributos
● Discretização
![Page 4: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/4.jpg)
![Page 5: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/5.jpg)
Big Data 6 Vs
• Volume• Variedade• Velocidade
• Valor• Variabilidade• Veracidade
5 29/09/14
V
![Page 6: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/6.jpg)
![Page 7: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/7.jpg)
Pré-Processamento
Vs
Big Data
![Page 8: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/8.jpg)
![Page 9: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/9.jpg)
![Page 10: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/10.jpg)
Open-Source
Desenvolvimento Acadêmico
Inovação
Novos Paradigmas
Critérios
![Page 11: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/11.jpg)
Critérios
![Page 12: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/12.jpg)
Batch X Real-time Processing
![Page 13: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/13.jpg)
Batch Processing
![Page 14: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/14.jpg)
![Page 15: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/15.jpg)
Hadoop (MapReduce)
![Page 16: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/16.jpg)
Hadoop (MapReduce)
![Page 17: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/17.jpg)
Hadoop
![Page 18: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/18.jpg)
HPCC SystemHPCC System
![Page 19: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/19.jpg)
HPCC SystemAPACHE DRILL
![Page 20: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/20.jpg)
HPCC SystemAPACHE DRILL
![Page 21: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/21.jpg)
HPCC SystemAPACHE DRILL
![Page 22: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/22.jpg)
HPCC SystemEcosystems
![Page 23: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/23.jpg)
APACHE SPARK
![Page 24: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/24.jpg)
APACHE SPARK
![Page 25: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/25.jpg)
APACHE SPARK
![Page 26: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/26.jpg)
Yahoo S4
![Page 27: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/27.jpg)
Apache Storm
![Page 28: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/28.jpg)
Apache Storm
![Page 29: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/29.jpg)
Apache Storm
![Page 30: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/30.jpg)
Apache Storm
![Page 31: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/31.jpg)
Apache Storm
![Page 32: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/32.jpg)
Apache Storm
![Page 33: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/33.jpg)
Apache Storm
![Page 34: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/34.jpg)
Apache Storm
![Page 35: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/35.jpg)
![Page 36: Pré-processamento em Big Data](https://reader034.fdocumentos.tips/reader034/viewer/2022050804/5483149db079591f0c8b4913/html5/thumbnails/36.jpg)
Considerações Finais
● Hot Topic
● Muitas ferramentas e frameworks disponíveis
● Importante conhecer o domínio
● Cuidado! Não existe chave mestra!
● Potencial em aprendizagem de máquina