[DOCS] Add an Expectation using the GX Cloud API#11567
[DOCS] Add an Expectation using the GX Cloud API#11567
Conversation
✅ Deploy Preview for niobium-lead-7998 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #11567 +/- ##
========================================
Coverage 84.18% 84.18%
========================================
Files 471 471
Lines 39592 39592
========================================
Hits 33330 33330
Misses 6262 6262 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
klavavej
left a comment
klavavej
left a comment
There was a problem hiding this comment.
Thanks for working on this!
I have one note that doesn’t fit in the diff:
- The Expectations overview says “Manually grouped into custom Expectation Suites via the GX Cloud API.” with the Expectation Suites link pointing to a Core page. Now that you’re adding instructions for populating Expectation Suites with the Cloud API, I think this link should be updated to point to the new Cloud content
Please note that there are a few places where I’ve left multiple inline comments on a single line to separate feedback where I’ve left a committable suggestion for the line from feedback that is more open ended for how you address it or needs content to be moved around to other lines entirely. We’ve discussed before how the GitHub UI can make these stacked comments easy to overlook so I wanted to give you a heads up about some of the review comments having this structure.
Papertrail:
Here are some issues for following up on this work in the future. TBD who will work these and when but I wanted to connect the dots for now
- Dynamic parameters and forecasted ranges - https://greatexpectations.atlassian.net/browse/DOC-1259
- size of code block font in collapsible - https://greatexpectations.atlassian.net/browse/DOC-1258
docs/docusaurus/docs/cloud/expectations/examples/create_an_expectation.py
Show resolved
Hide resolved
docs/docusaurus/docs/cloud/expectations/examples/create_an_expectation.py
Outdated
Show resolved
Hide resolved
| This is an example script for creating an expectation with preset or runtime parameters. | ||
|
|
||
| To test, run: | ||
| pytest --docs-tests -k "doc_example_create_an_expectation" tests/integration/test_script_runner.py |
There was a problem hiding this comment.
it looks like this test name is copied from the related Core code sample file. I think this will probably need to be changed to a unique name when you put the new code sample under test in docs/docusaurus/docs/components/examples_under_test.py
docs/docusaurus/docs/cloud/expectations/expectations_overview.md
Outdated
Show resolved
Hide resolved
| In your code, you will find the classes for Expectations in the `expectations` module. Import the module: | ||
|
|
||
| ```python title="Python" | ||
| from great_expectations import expectations as gxe |
There was a problem hiding this comment.
is this import necessary to include? I don't see the shorthand gxe anywhere else in this PR
There was a problem hiding this comment.
Oops, you're right, I was just so used to explicitly importing this module.
docs/docusaurus/docs/cloud/expectations/examples/create_an_expectation.py
Show resolved
Hide resolved
Co-authored-by: Kristen Lavavej <35638702+klavavej@users.noreply.github.com>
klavavej
left a comment
klavavej
left a comment
There was a problem hiding this comment.
Thanks for the edits! The changes you made mostly look good. I’ve left a few notes about some further polish I’d like to see now that I see the edits in context as well as a few items from the first round that were missed. One item from the first round that was missed is this note that didn’t fit in the diff that I left in the review summary:
- The Expectations overview says “Manually grouped into custom Expectation Suites via the GX Cloud API.” with the Expectation Suites link pointing to a Core page. Now that you’re adding instructions for populating Expectation Suites with the Cloud API, I think this link should be updated to point to the new Cloud content
| | Numeric | **column KL divergence to be less than*** | Expect the Kulback-Leibler (KL) divergence (relative entropy) of the specified column with respect to the partition object to be lower than the provided threshold. | No | No | | ||
| | Numeric | **column quantile values to be between*** | Expect the specific provided column quantiles to be between a minimum value and a maximum value. | No | No | |
There was a problem hiding this comment.
Thanks for filling in the empty cells.
These two Expectations still need to be moved to be in alphabetical order by name within the "Numeric" section per this note from the first round of review
|
|
||
| - [Python version 3.10 to 3.13](https://www.python.org/downloads/). | ||
| - [An installation of the Great Expectations Python library](https://pypi.org/project/great-expectations/). | ||
| - [A Data Context connected to your GX Cloud organization](/cloud/connect/connect_python.md#create-a-data-context). |
There was a problem hiding this comment.
Thanks for making this list have consistent punctuation. I still think this link lands too low and should be replaced with separate links for getting and setting credentials and a numbered step for getting the context per this note from the first round of review
| # Create an Expectation for Cloud | ||
| IntegrationTestFixture( | ||
| # To test, run: | ||
| # pytest --docs-tests -k "doc_example_create_an_expectation" tests/integration/test_script_runner.py |
There was a problem hiding this comment.
Thanks for adding the new code sample to this file. The name in this line looks like leftover copy/pasta that should be updated to reflect the unique name of the test in the line below.
| # pytest --docs-tests -k "doc_example_create_an_expectation" tests/integration/test_script_runner.py | |
| # pytest --docs-tests -k "doc_example_create_an_expectation_for_cloud" tests/integration/test_script_runner.py |
| This page provides instructions for working with Expectations. To learn about Expectation types and options, see the [Expectations overview](/cloud/expectations/expectations_overview.md). To learn about the Expectation changelog at the Data Asset level, visit [Manage Data Assets](/cloud/data_assets/manage_data_assets.md#view-data-asset-history). | ||
|
|
||
| - You have a [Data Asset](/cloud/data_assets/manage_data_assets.md). | ||
| ## Create an Expectation |
There was a problem hiding this comment.
Thanks for updating most instances of links to the old header to point to the new anchor. There's one lingering instance of a jump link within this page that still needs to be updated
Under https://deploy-preview-11567.docs.greatexpectations.io/docs/cloud/expectations/manage_expectations#generate-sql - the link to add a custom SQL Expectation still points to #add-an-expectation and should be updated to point to #create-an-expectation
| ### Next Steps | ||
|
|
||
| If you have created a new Expectation Suite, you will need to associate it to a Data Asset before you can run Validations. Visit [Run Validations](/docs/cloud/validations/run_validations/#api-managed-expectations-entire-asset) to learn how to do so. |
There was a problem hiding this comment.
Thanks for resolving the issue with the duplicate "prerequisite" headers in the table of contents.
I'm now noticing that, as a result of the same root cause of tabbed headers and the TOC not playing nice together, this "Next steps" header confusingly displays when the UI tab is selected though that content variant does not have a "Next steps" section
I suggest removing this header to resolve the issue.
Also, I still think the contents of this section are phrased in a misleading way as noted in the first round of review
Here's a suggestion to resolve both of these items
| ### Next Steps | |
| If you have created a new Expectation Suite, you will need to associate it to a Data Asset before you can run Validations. Visit [Run Validations](/docs/cloud/validations/run_validations/#api-managed-expectations-entire-asset) to learn how to do so. | |
| 7. If your Expectation Suite is not yet associated with a Data Asset through a Validation Definition, you must set up this relationship before you can validate your Expectations. Visit [Run Validations](/docs/cloud/validations/run_validations/#api-managed-expectations-entire-asset) to learn how to do so. |
| - [An installation of the Great Expectations Python library](https://pypi.org/project/great-expectations/). | ||
| - [A Data Context connected to your GX Cloud organization](/cloud/connect/connect_python.md#create-a-data-context). | ||
|
|
||
| 1. Choose an Expectation to create. |
There was a problem hiding this comment.
Thanks for removing the prereqs header in favor of prose introducing the prereqs. Now that I see it, I think it looks a bit funny to go from a bulleted list to a numbered list with no new list introduction between.
Here's a suggestion for a short introduction for the numbered list
| 1. Choose an Expectation to create. | |
| Follow the steps below to configure an Expectation and add it to an Expectation Suite: | |
| 1. Choose an Expectation to create. |
| There’s nowhere the need for iteration is more constant than when it comes to creating and configuring your data quality tests: as you uncover more nuances of your data, your operational needs change, or your data stack evolves, you’ll need to continue updating your Expectations. | ||
|
|
||
| Your testing feedback loop stays short and responsive in GX Cloud. Its visual GUI for Expectation editing makes it [a snap to create and edit them for nontechnical teams, while data engineers can use the power of GX Cloud’s API](/cloud/expectations/manage_expectations.md#add-an-expectation). | ||
| Your testing feedback loop stays short and responsive in GX Cloud. Its visual GUI for Expectation editing makes it [a snap to create and edit them for nontechnical teams, while data engineers can use the power of GX Cloud’s API](/cloud/expectations/manage_expectations.md#create-an-expectation). |
There was a problem hiding this comment.
This anchor link in the old 0.18 versioned docs should not have been updated. This change actually breaks the link since the destination header was not updated in the old 0.18 content
suggestion to undo this change:
| Your testing feedback loop stays short and responsive in GX Cloud. Its visual GUI for Expectation editing makes it [a snap to create and edit them for nontechnical teams, while data engineers can use the power of GX Cloud’s API](/cloud/expectations/manage_expectations.md#create-an-expectation). | |
| Your testing feedback loop stays short and responsive in GX Cloud. Its visual GUI for Expectation editing makes it [a snap to create and edit them for nontechnical teams, while data engineers can use the power of GX Cloud’s API](/cloud/expectations/manage_expectations.md#add-an-expectation). |
2 participants
Resolves DOC-1187
invoke lint(usesruff format+ruff check)For more information about contributing, visit our community resources.
After you submit your PR, keep the page open and monitor the statuses of the various checks made by our continuous integration process at the bottom of the page. Please fix any issues that come up and reach out on Slack if you need help. Thanks for contributing!