Misplaced Pages

Kubeflow

Article snapshot taken from Wikipedia with creative commons attribution-sharealike license. Give it a read and then ask your questions in the chat. We can research this topic together.
Open-source machine learning platform
A major contributor to this article appears to have a close connection with its subject. It may require cleanup to comply with Misplaced Pages's content policies, particularly neutral point of view. Please discuss further on the talk page. (September 2022) (Learn how and when to remove this message)
Kubeflow
Original author(s)Google
Developer(s)Kubeflow Contributors - AWS, Bloomberg, Google, IBM, NVIDIA, Nutanix, Red Hat, Arrikto, and others
Initial releaseApril 5, 2018; 6 years ago (2018-04-05)
Stable release1.9 / July 22, 2024; 5 months ago (2024-07-22)
Repositorygithub.com/kubeflow
Written inGo, Python, TypeScript
PlatformKubernetes
TypeMachine Learning Platform
LicenseApache License 2.0
Websitekubeflow.org

Kubeflow is an open-source platform for machine learning and MLOps on Kubernetes introduced by Google. The different stages in a typical machine learning lifecycle are represented with different software components in Kubeflow, including model development (Kubeflow Notebooks), model training (Kubeflow Pipelines, Kubeflow Training Operator), model serving (KServe), and automated machine learning (Katib).

Each component of Kubeflow can be deployed separately, and it is not a requirement to deploy every component.

History

The Kubeflow project was first announced at KubeCon + CloudNativeCon North America 2017 by Google engineers David Aronchick, Jeremy Lewi, and Vishnu Kannan to address a perceived lack of flexible options for building production-ready machine learning systems. The project has also stated it began as a way for Google to open-source how they ran TensorFlow internally.

The first release of Kubeflow (Kubeflow 0.1) was announced at KubeCon + CloudNativeCon Europe 2018. Kubeflow 1.0 was released in March 2020 via a public blog post announcing that many Kubeflow components were graduating to a "stable status", indicating they were now ready for production usage.

In October 2022, Google announced that the Kubeflow project had applied to join the Cloud Native Computing Foundation. In July 2023, the foundation voted to accept Kubeflow as an incubating stage project.

Components

Kubeflow Notebooks for model development

Machine learning models are developed in the notebooks component called Kubeflow Notebooks. The component runs web-based development environments inside a Kubernetes cluster, with native support for Jupyter Notebook, Visual Studio Code, and RStudio.

Kubeflow Pipelines for model training

Once developed, models are trained in the Kubeflow Pipelines component. The component acts as a platform for building and deploying portable, scalable machine learning workflows based on Docker containers. Google Cloud Platform has adopted the Kubeflow Pipelines DSL within its Vertex AI Pipelines product.

Kubeflow Training Operator for model training

For certain machine learning models and libraries, the Kubeflow Training Operator component provides Kubernetes custom resources support. The component runs distributed or non-distributed TensorFlow, PyTorch, Apache MXNet, XGBoost, and MPI training jobs on Kubernetes.

KServe for model serving

The KServe component (previously named KFServing) provides Kubernetes custom resources for serving machine learning models on arbitrary frameworks including TensorFlow, XGBoost, scikit-learn, PyTorch, and ONNX. KServe was developed collaboratively by Google, IBM, Bloomberg, NVIDIA, and Seldon. Publicly disclosed adopters of KServe include Bloomberg, Gojek, the Wikimedia Foundation, and others.

Katib for automated machine learning

Lastly, Kubeflow includes a component for automated training and development of machine learning models, the Katib component. It is described as a Kubernetes-native project and features hyperparameter tuning, early stopping, and neural architecture search.

Release timeline

Release timeline
Version Release Date Release Information Release Blog
Kubeflow 0.1 5 April, 2018 - https://kubernetes.io/blog/2018/05/04/announcing-kubeflow-0.1/
Kubeflow 0.2 2 July, 2018 - https://medium.com/kubeflow/kubeflow-0-2-offers-new-components-and-simplified-setup-735e4c56988d
Kubeflow 0.3 5 October, 2018 - https://medium.com/kubeflow/kubeflow-0-3-simplifies-setup-improves-ml-development-98b8ca10bd69
Kubeflow 0.4 8 January, 2019 - https://medium.com/kubeflow/kubeflow-0-4-release-enhancements-for-machine-learning-productivity-d77c54df07a9
Kubeflow 0.5 9 April, 2019 - https://medium.com/kubeflow/kubeflow-v0-5-simplifies-model-development-with-enhanced-ui-and-fairing-library-78e19cdc9f50
Kubeflow 0.6 19 July, 2019 https://www.kubeflow.org/docs/releases/kubeflow-0.6/ https://medium.com/kubeflow/kubeflow-v0-6-a-robust-foundation-for-artifact-tracking-data-versioning-multi-user-support-9896d329412c
Kubeflow 0.7 17 October, 2019 https://www.kubeflow.org/docs/releases/kubeflow-0.7/ https://medium.com/kubeflow/kubeflow-v0-7-delivers-beta-functionality-in-the-leadup-to-v1-0-1e63036c07b8
Kubeflow 1.0 20 February, 2020 https://www.kubeflow.org/docs/releases/kubeflow-1.0/ https://blog.kubeflow.org/releases/2020/03/02/kubeflow-1-0-cloud-native-ml-for-everyone
Kubeflow 1.1 31 July, 2020 https://www.kubeflow.org/docs/releases/kubeflow-1.1/ https://blog.kubeflow.org/release/official/2020/07/31/kubeflow-1.1-blog-post
Kubeflow 1.2 18 November, 2020 https://www.kubeflow.org/docs/releases/kubeflow-1.2/ https://blog.kubeflow.org/release/official/2020/11/18/kubeflow-1.2-blog-post
Kubeflow 1.3 23 April, 2021 https://www.kubeflow.org/docs/releases/kubeflow-1.3/ https://blog.kubeflow.org/kubeflow-1.3-release/
Kubeflow 1.4 12 October, 2021 https://www.kubeflow.org/docs/releases/kubeflow-1.4/ https://blog.kubeflow.org/kubeflow-1.4-release/
Kubeflow 1.5 10 March, 2022 https://www.kubeflow.org/docs/releases/kubeflow-1.5/ https://blog.kubeflow.org/kubeflow-1.5-release/
Kubeflow 1.6 7 September, 2022 https://www.kubeflow.org/docs/releases/kubeflow-1.6/ https://blog.kubeflow.org/kubeflow-1.6-release/
Kubeflow 1.7 29 March, 2023 https://www.kubeflow.org/docs/releases/kubeflow-1.7/ https://blog.kubeflow.org/kubeflow-1.7-release/
Kubeflow 1.8 1 November, 2023 https://www.kubeflow.org/docs/releases/kubeflow-1.8/ https://blog.kubeflow.org/kubeflow-1.8-release/
Kubeflow 1.9 22 July, 2024 https://www.kubeflow.org/docs/releases/kubeflow-1.9/ https://blog.kubeflow.org/kubeflow-1.9-release/

Notes

  1. KServe was previously known as KFServing

References

  1. "Kubeflow Website - Working Groups".
  2. ^ "Kubeflow 0.1 - Release Tag". GitHub.
  3. ^ "Kubeflow 1.9 - Release Information".
  4. "Kubeflow Website - Kubeflow Notebooks".
  5. "Kubeflow Website - Kubeflow Pipelines".
  6. ^ "Kubeflow GitHub - Kubeflow Training Operator". GitHub.
  7. "Kubeflow Website - KServe".
  8. "Kubeflow Website - Katib".
  9. "Kubeflow Website - Installing Kubeflow".
  10. ""Hot Dogs or Not" - At Scale with Kubernetes [I] - Vish Kannan & David Aronchick, Google". YouTube. 15 December 2017.
  11. "Introducing Kubeflow - A Composable, Portable, Scalable ML Stack Built for Kubernetes". 21 December 2017.
  12. "Kubeflow Website - History".
  13. "Google-led Kubeflow, machine learning for Kubernetes, begins to take shape". 4 May 2018.
  14. "Announcing Kubeflow 0.1". 4 May 2018.
  15. "Kubeflow 1.0: Cloud-Native ML for Everyone". 2 March 2020.
  16. Lamkin, Thea (2022-10-24). "Kubeflow has applied to become a CNCF incubating project". Kubeflow. Retrieved 2023-11-02.
  17. "Kubeflow applies to become a CNCF incubating project". Google Open Source Blog. 2022-10-24. Retrieved 2023-11-02.
  18. "Kubeflow brings MLOps to the CNCF Incubator". Cloud Native Computing Foundation. 2023-07-25. Retrieved 2023-11-02.
  19. "Kubeflow joins the CNCF family". Google Open Source Blog. 2023-07-25. Retrieved 2023-11-02.
  20. "Kubeflow Website - Kubeflow Notebooks Overview".
  21. "Kubeflow Website - Kubeflow Pipelines Introduction".
  22. "Vertex AI - Building a pipeline".
  23. ^ "KServe: The next generation of KFServing". 27 September 2021.
  24. "KServe GitHub". GitHub.
  25. "The journey to build Bloomberg's ML Inference Platform Using KServe (formerly KFServing)". Bloomberg L.p. 12 October 2021.
  26. "Merlin: Making ML Model Deployments Magical".
  27. "Machine Learning/LiftWing".
  28. "KServe Website - Adopters of KServe".
  29. "Kubeflow GitHub - Katib". GitHub.
  30. "Kubeflow 0.2 - Release Tag". GitHub.
  31. "Kubeflow 0.3 - Release Tag". GitHub.
  32. "Kubeflow 0.4 - Release Tag". GitHub.
  33. "Kubeflow 0.5 - Release Tag". GitHub.
  34. "Kubeflow 0.6 - Release Information".
  35. "Kubeflow 0.7 - Release Information".
  36. "Kubeflow 1.0 - Release Information".
  37. "Kubeflow 1.1 - Release Information".
  38. "Kubeflow 1.2 - Release Information".
  39. "Kubeflow 1.3 - Release Information".
  40. "Kubeflow 1.4 - Release Information".
  41. "Kubeflow 1.5 - Release Information".
  42. "Kubeflow 1.6 - Release Information".
  43. "Kubeflow 1.7 - Release Information".
  44. "Kubeflow 1.8 - Release Information".

External links

Google free and open-source software
Software
Applications
Programming languages
Frameworks and
development tools
Operating systems
Related
Categories: