Globalcode – Open4education

Soluções de arquitetura de dados utilizando Python

Rafael Calixto

Globalcode – Open4education

About meUFRJ

Licenciatura em Música

UEZO

Análise e Desenvolvimento de Sistemas

IGTI

Ciência de Dados (Big Data)

Wise&Trust

Cientista de Dados

PanoramaCrypto

Redator

Globalcode – Open4education

Cenário da arquitetura de dados

Web Crawlers

Analytics / Data Process

API

Data Ingestion

Globalcode – Open4education

Linguagens mais usadas em projetos no Github

https://www.benfrederickson.com/ranking-programming-languages-by-github-users/

Globalcode – Open4education

Web Crawlers

Diversos frameworks e bibliotecas disponíveis:

Globalcode – Open4education

Analytics / Data Process

Bibliotecas para processamento de dados:

Globalcode – Open4education

APIs

Frameworks para a construção de APIs com python:

Globalcode – Open4education

Conexão com Bases de Dados

Globalcode – Open4education

Case Datarisk.io

Criando filas no Redis

Globalcode – Open4education

Pipeline da Datarisk.io

request

status

add to queue

data process

output

input

add to queue

front end

Globalcode – Open4education

Case Wise&Trust

Trading data

Globalcode – Open4education

Pipeline da Wise&Trustcrawlers data process

relational storage

data analytics

applicationssemi-structured storage

psycopg2pymongo

Globalcode – Open4education

Perguntas?

Globalcode – Open4education

ContatosTwitter@rafaelcalixtopy

LinkedInhttps://www.linkedin.com/in/rafael-calixto-9a11936b/

Githubhttps://github.com/rafaelcalixto

Telegram@rafaelcalixto