Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TFOperator support in FlytePlugins #115

Open
kumare3 opened this issue Nov 22, 2019 · 6 comments
Open

TFOperator support in FlytePlugins #115

kumare3 opened this issue Nov 22, 2019 · 6 comments

Comments

@kumare3
Copy link
Collaborator

@kumare3 kumare3 commented Nov 22, 2019

TL;DR
Add support for new "tensorflow" task type, that uses TFOperator under the hood.

Summary:
Flyte allows orchestrating across different orchestrators. Though a simple TF job could be python code, using TFOperator allows other benefits. Namely Tensorboard and ability to run distributed tf jobs.

Operator to use:
https://github.com/kubeflow/tf-operator

Components:

  • Golang Flyteplugins client for TFOperator
  • Protobuf schema for operator metadata
  • Input / Output unmarshalling
  • Python flytekit task definition for Tensorflow jobs
@kumare3
Copy link
Collaborator Author

@kumare3 kumare3 commented Nov 22, 2019

cc @swiftdiaries is interested in doing this
https://github.com/swiftdiaries

@swiftdiaries
Copy link

@swiftdiaries swiftdiaries commented Nov 22, 2019

Ack :) I'll start work on this next week

@kumare3
Copy link
Collaborator Author

@kumare3 kumare3 commented Jun 16, 2020

@igorvalko recently contributed pytorch-operator plugin for Flyte. The issue is #338

@kumare3
Copy link
Collaborator Author

@kumare3 kumare3 commented Jun 16, 2020

I have added all of his PRs to the issue - #338 (comment)

@kumare3
Copy link
Collaborator Author

@kumare3 kumare3 commented Jun 30, 2020

https://github.com/pingsutw has volunteered to help with this

@kumare3
Copy link
Collaborator Author

@kumare3 kumare3 commented Jun 30, 2020

Work in progress at - https://github.com/flyteorg/kfplugins

@pingsutw pingsutw self-assigned this Jul 2, 2020
@pingsutw pingsutw mentioned this issue Jul 22, 2020
5 of 8 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.