Netflix Cosmos Platform

Netflix Cosmos is a computing platform that combines the very best aspects of microservices, as well as support for asynchronous workflows and serverless functionality. Our strong point is the development of applications that implement resource-intensive algorithms that are coordinated using a complex hierarchy of workflows and cover different time periods - from a few minutes to several years. The platform enables not only high-throughput services with hundreds of thousands of processors in parallel, but also latency-sensitive workloads where it is important to show results to users as quickly as possible.





Netflix Cosmos service
Netflix Cosmos service

, Netflix Cosmos, ,  , .





Netflix , , β€” Media Cloud Engineering Encoding Technologies. 2007 . . .  β€” Reloaded β€” . .





Reloaded . : . , , -, . . , . , . , , .





Netflix Cosmos β€” , . , , :





  •  β€” , , , , , ;





  •  β€” , , ;





  •  β€” , , , ;





  •  β€” , , . - .





, , .





Netflix Cosmos , .  β€” API‑, - (stateless) . API‑ , .





Regular microservice

Netflix Cosmos , . Netflix Cosmos API‑ . . , . Docker (, Debian) .     . .





Netflix Cosmos regular service
Netflix Cosmos

Netflix Cosmos . , API‑, . , . API‑ , . , , : API‑, . , . Netflix Cosmos, :





  • Optimus β€” API‑, ‑;





  • Plato β€” , ‑;





  • Stratum β€” , stateless‑.





Timestone β€” . . , Netflix Cosmos.





Separation between platform and application.
.

Netflix Cosmos

Netflix Cosmos service request trace graph.
Netflix Cosmos.

Nirvana β€” , . Netflix Cosmos (  β€”   ):





  1. API‑ .





  2. 31 , 31 .





  3. .





  4. .





  5. .





Netflix Cosmos . , API‑ .





, , , β€” , . , , , , .  β€” Tapas. , Netflix.  β€” Sagan. ,   .





Netflix Cosmos service levels.
Netflix Cosmos.

- , Tapas, , :





  • ;





  •   , ;





  • ;





  •   ;





  • .





  • ;





  •   , ;





  • ;





  •   ;





  • .





, Tapas Netflix Cosmos Stratum.





, , . 24 , 8 Netflix Cosmos 9 Stratum.





Service request trace traffic at multiple levels.
.

!

, Β« Β»? Plato β€” , Netflix Cosmos, . API Optimus . Stratum RPC, serverless-.





Plato β€” , . , Netflix Conductor, Plato Β« Β». , . , , , .





Plato Apache Karaf, . , Plato.





, Emirax, Groovy. :





  • match β€” , ;





  • action β€” , ( Stratum );





  • reaction β€” , action;





  • error β€” , .





, Stratum ( β€”  feathercast.apache.org).





,

Netflix Cosmos, Sagan, , . , , , , Β« Β». Stratum , , , . ( ) . . ,  β€” . , , : - 30 .





Lack of resources.
.

 β€” , . 30 , . Netflix Cosmos ( , Stratum) . Stratum :





  •  β€” Stratum . , .





  •  β€” (, ) , Stratum.





  •  β€” , Apache Spark, . , . 10 000 , 10 000 10 1000 .





  •  β€” , Netflix Cosmos - : , , . , , . Netflix Cosmos , API .





,

, Tapas, , ( ). , . , (SLO) , .





, , SLO β€” , Stratum. Stratum Titus «» . , serverless- , .





-

, , , Reloaded, , . , . , - β€” .





Netflix Cosmos 2018 2019 . 40 Netflix Cosmos , . , , .





Netflix

Netflix , Β« Β». , . β€” . Netflix Cosmos . Optimus, Plato Stratum . API‑ . - . .





 +  +

, Β« , serverless-Β» β€” . . , .





Β« Β» . . - ,  . .   , , . - , . , , . . , , , , , .





Netflix Cosmos, , -  β€” . , , . Netflix Cosmos , , , , . , , , .





2021 will be a big year for Netflix Cosmos as we move most of our work from Reloaded to Cosmos, add additional developers to the project, and ramp up infrastructure workloads. We plan to develop our programming model to include new use cases. Our goal is to make Netflix Cosmos easier to use, more sustainable, faster and more efficient. Subscribe to our blog for more details on how Netflix Cosmos works and how we use the framework.








All Articles