Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(docs): refactor source and sink docs #3031

Merged
merged 40 commits into from
Aug 8, 2021

Conversation

kevinhu
Copy link
Contributor

@kevinhu kevinhu commented Aug 4, 2021

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable)


Note that a `.` is used to denote nested fields in the YAML recipe.

| Field | Required | Default | Description |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is amazing.

Copy link
Collaborator

@jjoyce0510 jjoyce0510 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First of all : WOW! This was a huge amount of work to make these docs better. Thank you so much for your efforts! These changes will greatly improve the usability of the ingestion framework.

Now for a summary of comments. Overall it looks great, just a couple suggestions remaining.

I see that you've removed metadata-ingestion README. I do think we still need a top level parent page for

  • What is the metadata ingestion framework
  • What is a recipe file
  • How am I expected to productionize / deploy this?
  • Any links to contributing guides etc.

I think we should continue to try to establish conventions for common patterns in each source doc:

  • credentials / authentication
  • pointing to the correct location of the source system
  • and connector-specific options and feature configurations
    It'd be useful to think about ways to highlight these categories in a consistent manner across all source docs!

Copy link
Collaborator

@jjoyce0510 jjoyce0510 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Thank you!

Copy link
Contributor

@shirshanka shirshanka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks a lot for this awesome contrib @kevinhu

@shirshanka shirshanka merged commit 32b8fc6 into datahub-project:master Aug 8, 2021
rahulbsw pushed a commit to rahulbsw/datahub that referenced this pull request Sep 2, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants