# Netflix Blog- Netflix Python技术的使用
Date: 2019-05-21


这篇文章  [Python at Netflix](https://medium.com/netflix-techblog/python-at-netflix-bba45dae649e)  是出自 [Netflix Technology Blog](https://medium.com/@NetflixTechBlog)

本文主要介绍python在Netflix各个组内的应用

1. #### Open Connect:
    
    - [Netflix搭建的全球CDN网络，](https://media.netflix.com/en/company-blog/how-netflix-works-with-isps-around-the-globe-to-deliver-a-great-viewing-experience)python有广泛应用。
2. ####  Demand Engineering：
    
    - 数值分析类：[numpy](https://www.numpy.org/)， [scipy](https://www.scipy.org/)
    - AWS SDK：[boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html) 
    - Web 框架： [Flask](http://flask.pocoo.org/)
    - 队列： [RQ](https://python-rq.org/)（Redis queue）
    - 轻量级IDE： [bpython](https://bpython-interpreter.org/)
    - 编辑工具： [Jupyter Notebooks](https://jupyter.org/)， [nteract](https://nteract.io/)
    - Data Explorer： [visualization tools](https://github.com/nteract/nteract/tree/master/packages/data-explorer)
    - Data FrameWork：[Semiotic](https://semiotic.nteract.io/)
3. #### CORE:
    
    - 统计学数值分析类：[numpy](https://www.numpy.org/)， [scipy](https://www.scipy.org/)      [ruptures](https://github.com/deepcharles/ruptures)    [pandas](https://pandas.pydata.org/)
4. #### Monitoring, alerting and auto-remediation:
    
    -  [Spectator](https://github.com/Netflix/spectator)   :  [metris](https://github.com/dropwizard/metrics)的客户端
        - 客户端： [Spectator Python client](https://github.com/Netflix/spectator-py) 
    -  [Winston](https://medium.com/netflix-techblog/introducing-winston-event-driven-diagnostic-and-remediation-platform-46ce39aa81cc) and [Bolt](https://medium.com/netflix-techblog/introducing-bolt-on-instance-diagnostic-and-remediation-platform-176651b55505)  :
        - Web 框架： [Flask](http://flask.pocoo.org/)
        - WSGI http server： [Gunicorn](https://gunicorn.org/) 
        - Flask Rest APIs  插件： [Flask-RESTPlus](https://github.com/noirbizarre/flask-restplus)
5. #### Information Security：
    
    - 安全监控：   [Security Monkey](https://github.com/Netflix/security_monkey)
    - SSH资源保护： [Bless](https://github.com/Netflix/bless)
    - TLS证书生成：[Lemur](https://github.com/Netflix/lemur).
    - AWS安全部署（权限最小化）： [Aardvark](https://github.com/Netflix-Skunkworks/aardvark) and [Repokid](https://github.com/Netflix/Repokid)
    - 数据取证 ：  [Diffy](https://medium.com/netflix-techblog/netflix-sirt-releases-diffy-a-differencing-engine-for-digital-forensics-in-the-cloud-37b71abd2698)   Prism，  Lanius
6. #### Personalization Algorithms：
    
    - 深度神经网络学习： [TensorFlow](https://github.com/tensorflow)  [Keras](https://github.com/keras-team/keras)  [PyTorch](https://pytorch.org/)
    - Gradient Boosted Decision Trees ： [XGBoost](https://github.com/dmlc/xgboost)    [LightGBM](https://github.com/microsoft/LightGBM)
    - 其他相关科学运算类：[numpy](https://www.numpy.org/)， [scipy](https://www.scipy.org/)   [pandas](https://pandas.pydata.org/)  [sklearn](https://scikit-learn.org/stable/)  [matplotlib](https://matplotlib.org/) [cvxpy](https://github.com/cvxgrp/cvxpy)
7. #### Machine Learning Infrastructure：
    
    - ML Libraries: [R Project](https://www.r-project.org/)   [TensorFlow](https://github.com/tensorflow)    [XGBoost](https://github.com/dmlc/xgboost)
    - NoteBooks:  [nteract](https://nteract.io/)
    - Job Schedule: Meson  基于Mesos开发的工具，类似与[Airflow](https://airflow.apache.org/)
        
        [Meson: Workflow Orchestration for Netflix Recommendations](https://medium.com/netflix-techblog/meson-workflow-orchestration-for-netflix-recommendations-fc932625c1d9)
        
    - Computer Resource： [Titus](https://github.com/Netflix/titus)   类似于kubernetes
    - Query Engine ：Spark
    - Data Lake ： S3
8. #### Notebook：
    
    - [Jupyter Notebooks](https://jupyter.org/)，
    - [blog：Beyond Interactive: Notebook Innovation at Netflix](https://medium.com/netflix-techblog/notebook-innovation-591ee3221233)
        
9. #### Orchestration：
    
    - 工具： [Jupyter Notebooks](https://jupyter.org/)     with  [papermill](https://papermill.readthedocs.io/en/latest/)
    - blog：[Part 2: Scheduling Notebooks at Netflix](https://medium.com/netflix-techblog/scheduling-notebooks-348e6c14cfd6)
    - [Genie](https://netflix.github.io/genie/)Client：  [pygenie](https://github.com/Netflix/pygenie)
    - other：[scrapbook](https://nteract-scrapbook.readthedocs.io/en/latest/)
10. #### Experimentation Platform：
     
     - （Metrics库）The Metrics Repo ： 使用了[PyPika](https://pypika.readthedocs.io/en/latest/)   
     - （因果关系）The Causal Models library： 使用了  [PyArrow](https://arrow.apache.org/docs/python/) and [RPy2](https://rpy2.readthedocs.io/en/version_2.8.x/)
     - （可视化）The Visualizations library： 使用了 [Plotly](https://plot.ly/)
11. #### Partner Ecosystem：
     
     - CI infrastructure
     - test
12. #### Video Encoding and Media Cloud Engineering：
     
     - 涉及项目：[vmaf](https://github.com/Netflix/vmaf/blob/master/resource/doc/references.md) and [mezzfs](https://medium.com/netflix-techblog/mezzfs-mounting-object-storage-in-netflixs-media-processing-platform-cda01c446ba)
     - 平台： a media map-reduce platform called [Archer](https://medium.com/netflix-techblog/simplifying-media-innovation-at-netflix-with-archer-3f8cbb0e2bcb),
     - 工具： [setupmeta](https://pypi.org/project/setupmeta/) and [pickley](https://pypi.org/project/pickley/).
13. #### Netflix Animation and NVFX（视觉效果）：
     
     - 外部对接 与Maya and Nuke 相关业务集成使用python
     - 工具： Shotgun
     - 管理镜像： AMI（in EC2），  Docker
14. #### Content Machine Learning, Science & Analytics：
     
     -  machine learning models

#### 参考：

[Metrics 是个什么鬼 之入门教程](http://wuchong.me/blog/2015/08/01/getting-started-with-metrics/)

[difference-between-docker-and-ami](https://stackoverflow.com/questions/41686811/difference-between-docker-and-ami)

[Netflix 实践：1.48 亿用户观看电影，Python 在幕后提供全生命周期支持](https://www.infoq.cn/article/x6Bkrx-BbvmYcfO24RcC)