Skip to content

/ vespa

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support feeding from Spark #9158

Open
lesters opened this issue Apr 23, 2019 · 0 comments
Open

Support feeding from Spark #9158

lesters opened this issue Apr 23, 2019 · 0 comments
Labels
enhancement good first issue
Milestone
later

Comments

@lesters
Copy link
Contributor

@lesters lesters commented Apr 23, 2019

Today, the Hadoop integration tools for Vespa support Hadoop and Pig for feeding and querying Vespa. The Pig feeder is a thin wrapper around the Vespa HTTP client.

We should support feeding directly from Spark as well, to avoid Spark pipelines having to write to HDFS and run another Pig job for the actual feeding. Similarly to the Pig feeder, this could be implemented as a thin wrapper around the HTTP client.
@lesters lesters added the enhancement label Apr 23, 2019
@frodelu frodelu added this to the later milestone Apr 24, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement good first issue
Projects
None yet
Milestone
  later
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
@frodelu @lesters
You can’t perform that action at this time.