Add OSS sensor backcompat tests #5889

dpeng817 · 2021-12-07T02:52:39Z

The current backcompat tests on OSS only test basic pipeline execution and retrieval functionality. This adds sensor start, stop, and evaluation as well.

Had to bump the minimum version for backcompat tests, because sensor backcompat is broken on 0.12.8.

vercel · 2021-12-07T02:52:42Z

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployments, click below or on the icon next to each commit.

dagit-storybook – ./js_modules/dagit/packages/core

🔍 Inspect: https://vercel.com/elementl/dagit-storybook/6V88HNReesw7MBqR3pEKSDJeaSPU
✅ Preview: Canceled

dagster – ./docs/next

🔍 Inspect: https://vercel.com/elementl/dagster/CwS4rj6cr8mE5efqs5rWJBMZjpP8
✅ Preview: Canceled

gibsondan · 2021-12-09T23:16:32Z

"Had to bump the minimum version for backcompat tests, because sensor backcompat is broken on 0.12.8." - do you have more info on this? I thought the main goal of these tests was to catch breaking changes between 0.12.x and 0.13.x

dpeng817 · 2021-12-09T23:39:08Z

"Had to bump the minimum version for backcompat tests, because sensor backcompat is broken on 0.12.8." - do you have more info on this? I thought the main goal of these tests was to catch breaking changes between 0.12.x and 0.13.x

#4745 - broke compat with versions before 0.12.11 when using sensors. If this test was around then, it would have caught this.

gibsondan

got it - I feel better about it now that there's still an 0.12.x as the earliest release

looks like the test suite is failing though on that release?

addressed

gibsondan · 2021-12-15T22:13:18Z

integration_tests/test_suites/backcompat-test-suite/test_backcompat.py

+ runs_list = []
+ while are_all_runs_complete(runs_list):
+ response = graphql_client._execute(RUNS_QUERY)
+ runs_list = response["pipelineRunsOrError"]["results"]
+
+ for run in runs_list:
+ assert run["status"] == "SUCCESS"


as discussed briefly offline this seems not right - we should probably wait until there is at least one run between START and STOP (as opposed to waiting 5 seconds), then here wait for them to all complete?

alangenfeld

to your queue for dgibsons comment

…r sensors)

dpeng817 · 2022-01-07T19:37:51Z

Running into a weird issue here where the stop sensor query doesn't seem to be actually stopping the sensor, resulting in it running forever and creating a ton of runs.

dpeng817 · 2022-01-26T17:18:51Z

Closing for now until I have time to revisit

Bumps [axios](https://github.com/axios/axios) from 0.26.0 to 1.6.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/axios/axios/releases">axios's releases</a>.</em></p> <blockquote> <h2>Release v1.6.0</h2> <h2>Release notes:</h2> <h3>Bug Fixes</h3> <ul> <li><strong>CSRF:</strong> fixed CSRF vulnerability CVE-2023-45857 (<a href="https://redirect.github.com/axios/axios/issues/6028">#6028</a>) (<a href="https://github.com/axios/axios/commit/96ee232bd3ee4de2e657333d4d2191cd389e14d0">96ee232</a>)</li> <li><strong>dns:</strong> fixed lookup function decorator to work properly in node v20; (<a href="https://redirect.github.com/axios/axios/issues/6011">#6011</a>) (<a href="https://github.com/axios/axios/commit/5aaff532a6b820bb9ab6a8cd0f77131b47e2adb8">5aaff53</a>)</li> <li><strong>types:</strong> fix AxiosHeaders types; (<a href="https://redirect.github.com/axios/axios/issues/5931">#5931</a>) (<a href="https://github.com/axios/axios/commit/a1c8ad008b3c13d53e135bbd0862587fb9d3fc09">a1c8ad0</a>)</li> </ul> <h3>PRs</h3> <ul> <li>CVE 2023 45857 ( <a href="https://api.github.com/repos/axios/axios/pulls/6028">#6028</a> )</li> </ul> <pre><code> ⚠️ Critical vulnerability fix. See https://security.snyk.io/vuln/SNYK-JS-AXIOS-6032459 </code></pre> <h3>Contributors to this release</h3> <ul> <li> <a href="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/DigitalBrainJS" title="+449/-114 ([#6032](axios/axios#6032) [#6021](axios/axios#6021) [#6011](axios/axios#6011) [#5932](axios/axios#5932) [#5931](axios/axios#5931) )">Dmitriy Mozgovoy</a></li> <li> <a href="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/valentin-panov" title="+4/-4 ([#6028](axios/axios#6028) )">Valentin Panov</a></li> <li> <a href="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/therealrinku" title="+1/-1 ([#5889](axios/axios#5889) )">Rinku Chaudhari</a></li> </ul> <h2>Release v1.5.1</h2> <h2>Release notes:</h2> <h3>Bug Fixes</h3> <ul> <li><strong>adapters:</strong> improved adapters loading logic to have clear error messages; (<a href="https://redirect.github.com/axios/axios/issues/5919">#5919</a>) (<a href="https://github.com/axios/axios/commit/e4107797a7a1376f6209fbecfbbce73d3faa7859">e410779</a>)</li> <li><strong>formdata:</strong> fixed automatic addition of the <code>Content-Type</code> header for FormData in non-browser environments; (<a href="https://redirect.github.com/axios/axios/issues/5917">#5917</a>) (<a href="https://github.com/axios/axios/commit/bc9af51b1886d1b3529617702f2a21a6c0ed5d92">bc9af51</a>)</li> <li><strong>headers:</strong> allow <code>content-encoding</code> header to handle case-insensitive values (<a href="https://redirect.github.com/axios/axios/issues/5890">#5890</a>) (<a href="https://redirect.github.com/axios/axios/issues/5892">#5892</a>) (<a href="https://github.com/axios/axios/commit/4c89f25196525e90a6e75eda9cb31ae0a2e18acd">4c89f25</a>)</li> <li><strong>types:</strong> removed duplicated code (<a href="https://github.com/axios/axios/commit/9e6205630e1c9cf863adf141c0edb9e6d8d4b149">9e62056</a>)</li> </ul> <h3>Contributors to this release</h3> <ul> <li> <a href="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/DigitalBrainJS" title="+89/-18 ([#5919](axios/axios#5919) [#5917](axios/axios#5917) )">Dmitriy Mozgovoy</a></li> <li> <a href="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/DavidJDallas" title="+11/-5 ()">David Dallas</a></li> <li> <a href="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/fb-sean" title="+2/-8 ()">Sean Sattler</a></li> <li> <a href="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/0o001" title="+4/-4 ()">Mustafa Ateş Uzun</a></li> <li> <a href="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/sfc-gh-pmotacki" title="+2/-1 ([#5892](axios/axios#5892) )">Przemyslaw Motacki</a></li> <li> <a href="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/Cadienvan" title="+1/-1 ()">Michael Di Prisco</a></li> </ul> <h2>Release v1.5.0</h2> <h2>Release notes:</h2> <h3>Bug Fixes</h3> <ul> <li><strong>adapter:</strong> make adapter loading error more clear by using platform-specific adapters explicitly (<a href="https://redirect.github.com/axios/axios/issues/5837">#5837</a>) (<a href="https://github.com/axios/axios/commit/9a414bb6c81796a95c6c7fe668637825458e8b6d">9a414bb</a>)</li> <li><strong>dns:</strong> fixed <code>cacheable-lookup</code> integration; (<a href="https://redirect.github.com/axios/axios/issues/5836">#5836</a>) (<a href="https://github.com/axios/axios/commit/b3e327dcc9277bdce34c7ef57beedf644b00d628">b3e327d</a>)</li> <li><strong>headers:</strong> added support for setting header names that overlap with class methods; (<a href="https://redirect.github.com/axios/axios/issues/5831">#5831</a>) (<a href="https://github.com/axios/axios/commit/d8b4ca0ea5f2f05efa4edfe1e7684593f9f68273">d8b4ca0</a>)</li> <li><strong>headers:</strong> fixed common Content-Type header merging; (<a href="https://redirect.github.com/axios/axios/issues/5832">#5832</a>) (<a href="https://github.com/axios/axios/commit/8fda2766b1e6bcb72c3fabc146223083ef13ce17">8fda276</a>)</li> </ul> <h3>Features</h3>  </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/axios/axios/blob/v1.x/CHANGELOG.md">axios's changelog</a>.</em></p> <blockquote> <h1><a href="https://github.com/axios/axios/compare/v1.5.1...v1.6.0">1.6.0</a> (2023-10-26)</h1> <h3>Bug Fixes</h3> <ul> <li><strong>CSRF:</strong> fixed CSRF vulnerability CVE-2023-45857 (<a href="https://redirect.github.com/axios/axios/issues/6028">#6028</a>) (<a href="https://github.com/axios/axios/commit/96ee232bd3ee4de2e657333d4d2191cd389e14d0">96ee232</a>)</li> <li><strong>dns:</strong> fixed lookup function decorator to work properly in node v20; (<a href="https://redirect.github.com/axios/axios/issues/6011">#6011</a>) (<a href="https://github.com/axios/axios/commit/5aaff532a6b820bb9ab6a8cd0f77131b47e2adb8">5aaff53</a>)</li> <li><strong>types:</strong> fix AxiosHeaders types; (<a href="https://redirect.github.com/axios/axios/issues/5931">#5931</a>) (<a href="https://github.com/axios/axios/commit/a1c8ad008b3c13d53e135bbd0862587fb9d3fc09">a1c8ad0</a>)</li> </ul> <h3>PRs</h3> <ul> <li>CVE 2023 45857 ( <a href="https://api.github.com/repos/axios/axios/pulls/6028">#6028</a> )</li> </ul> <pre><code> ⚠️ Critical vulnerability fix. See https://security.snyk.io/vuln/SNYK-JS-AXIOS-6032459 </code></pre> <h3>Contributors to this release</h3> <ul> <li> <a href="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/DigitalBrainJS" title="+449/-114 ([#6032](axios/axios#6032) [#6021](axios/axios#6021) [#6011](axios/axios#6011) [#5932](axios/axios#5932) [#5931](axios/axios#5931) )">Dmitriy Mozgovoy</a></li> <li> <a href="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/valentin-panov" title="+4/-4 ([#6028](axios/axios#6028) )">Valentin Panov</a></li> <li> <a href="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/therealrinku" title="+1/-1 ([#5889](axios/axios#5889) )">Rinku Chaudhari</a></li> </ul> <h2><a href="https://github.com/axios/axios/compare/v1.5.0...v1.5.1">1.5.1</a> (2023-09-26)</h2> <h3>Bug Fixes</h3> <ul> <li><strong>adapters:</strong> improved adapters loading logic to have clear error messages; (<a href="https://redirect.github.com/axios/axios/issues/5919">#5919</a>) (<a href="https://github.com/axios/axios/commit/e4107797a7a1376f6209fbecfbbce73d3faa7859">e410779</a>)</li> <li><strong>formdata:</strong> fixed automatic addition of the <code>Content-Type</code> header for FormData in non-browser environments; (<a href="https://redirect.github.com/axios/axios/issues/5917">#5917</a>) (<a href="https://github.com/axios/axios/commit/bc9af51b1886d1b3529617702f2a21a6c0ed5d92">bc9af51</a>)</li> <li><strong>headers:</strong> allow <code>content-encoding</code> header to handle case-insensitive values (<a href="https://redirect.github.com/axios/axios/issues/5890">#5890</a>) (<a href="https://redirect.github.com/axios/axios/issues/5892">#5892</a>) (<a href="https://github.com/axios/axios/commit/4c89f25196525e90a6e75eda9cb31ae0a2e18acd">4c89f25</a>)</li> <li><strong>types:</strong> removed duplicated code (<a href="https://github.com/axios/axios/commit/9e6205630e1c9cf863adf141c0edb9e6d8d4b149">9e62056</a>)</li> </ul> <h3>Contributors to this release</h3> <ul> <li> <a href="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/DigitalBrainJS" title="+89/-18 ([#5919](axios/axios#5919) [#5917](axios/axios#5917) )">Dmitriy Mozgovoy</a></li> <li> <a href="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/DavidJDallas" title="+11/-5 ()">David Dallas</a></li> <li> <a href="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/fb-sean" title="+2/-8 ()">Sean Sattler</a></li> <li> <a href="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/0o001" title="+4/-4 ()">Mustafa Ateş Uzun</a></li> <li> <a href="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/sfc-gh-pmotacki" title="+2/-1 ([#5892](axios/axios#5892) )">Przemyslaw Motacki</a></li> <li> <a href="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/Cadienvan" title="+1/-1 ()">Michael Di Prisco</a></li> </ul> <h3>PRs</h3> <ul> <li>CVE 2023 45857 ( <a href="https://api.github.com/repos/axios/axios/pulls/6028">#6028</a> )</li> </ul> <pre><code> ⚠️ Critical vulnerability fix. See https://security.snyk.io/vuln/SNYK-JS-AXIOS-6032459 </code></pre> <h1><a href="https://github.com/axios/axios/compare/v1.4.0...v1.5.0">1.5.0</a> (2023-08-26)</h1>  </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/axios/axios/commit/f7adacdbaa569281253c8cfc623ad3f4dc909c60"><code>f7adacd</code></a> chore(release): v1.6.0 (<a href="https://redirect.github.com/axios/axios/issues/6031">#6031</a>)</li> <li><a href="https://github.com/axios/axios/commit/9917e67cbb6c157382863bad8c741de58e3f3c2b"><code>9917e67</code></a> chore(ci): fix release-it arg; (<a href="https://redirect.github.com/axios/axios/issues/6032">#6032</a>)</li> <li><a href="https://github.com/axios/axios/commit/96ee232bd3ee4de2e657333d4d2191cd389e14d0"><code>96ee232</code></a> fix(CSRF): fixed CSRF vulnerability CVE-2023-45857 (<a href="https://redirect.github.com/axios/axios/issues/6028">#6028</a>)</li> <li><a href="https://github.com/axios/axios/commit/7d45ab2e2ad6e59f5475e39afd4b286b1f393fc0"><code>7d45ab2</code></a> chore(tests): fixed tests to pass in node v19 and v20 with <code>keep-alive</code> enabl...</li> <li><a href="https://github.com/axios/axios/commit/5aaff532a6b820bb9ab6a8cd0f77131b47e2adb8"><code>5aaff53</code></a> fix(dns): fixed lookup function decorator to work properly in node v20; (<a href="https://redirect.github.com/axios/axios/issues/6011">#6011</a>)</li> <li><a href="https://github.com/axios/axios/commit/a48a63ad823fc20e5a6a705f05f09842ca49f48c"><code>a48a63a</code></a> chore(docs): added AxiosHeaders docs; (<a href="https://redirect.github.com/axios/axios/issues/5932">#5932</a>)</li> <li><a href="https://github.com/axios/axios/commit/a1c8ad008b3c13d53e135bbd0862587fb9d3fc09"><code>a1c8ad0</code></a> fix(types): fix AxiosHeaders types; (<a href="https://redirect.github.com/axios/axios/issues/5931">#5931</a>)</li> <li><a href="https://github.com/axios/axios/commit/2ac731d60545ba5c4202c25fd2e732ddd8297d82"><code>2ac731d</code></a> chore(docs): update readme.md (<a href="https://redirect.github.com/axios/axios/issues/5889">#5889</a>)</li> <li><a href="https://github.com/axios/axios/commit/88fb52b5fad7aabab0532e7ad086c5f1b0178905"><code>88fb52b</code></a> chore(release): v1.5.1 (<a href="https://redirect.github.com/axios/axios/issues/5920">#5920</a>)</li> <li><a href="https://github.com/axios/axios/commit/e4107797a7a1376f6209fbecfbbce73d3faa7859"><code>e410779</code></a> fix(adapters): improved adapters loading logic to have clear error messages; ...</li> <li>Additional commits viewable in <a href="https://github.com/axios/axios/compare/v0.26.0...v1.6.0">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=axios&package-manager=npm_and_yarn&previous-version=0.26.0&new-version=1.6.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/dagster-io/dagster/network/alerts). </details> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

dpeng817 requested review from alangenfeld and prha December 7, 2021 02:52

dpeng817 force-pushed the sensor_backcompat_tests branch from e4699a4 to e431d54 Compare December 8, 2021 18:50

vercel bot temporarily deployed to Preview December 8, 2021 18:50 Inactive

dpeng817 force-pushed the sensor_backcompat_tests branch from e431d54 to f415fdf Compare December 8, 2021 19:00

vercel bot temporarily deployed to Preview December 8, 2021 19:00 Inactive

dpeng817 force-pushed the sensor_backcompat_tests branch from f415fdf to ca6c858 Compare December 9, 2021 21:42

vercel bot temporarily deployed to Preview December 9, 2021 21:42 Inactive

dpeng817 changed the title ~~[draft][rfc] Add OSS sensor backcompat tests~~ Add OSS sensor backcompat tests Dec 9, 2021

dpeng817 requested a review from gibsondan December 9, 2021 21:42

vercel bot temporarily deployed to Preview December 9, 2021 23:51 Inactive

gibsondan previously requested changes Dec 10, 2021

View reviewed changes

dpeng817 force-pushed the sensor_backcompat_tests branch from f37f72f to cbe8a05 Compare December 15, 2021 02:41

vercel bot temporarily deployed to Preview December 15, 2021 02:41 Inactive

dpeng817 requested a review from gibsondan December 15, 2021 16:44

gibsondan reviewed Dec 15, 2021

View reviewed changes

alangenfeld requested a review from gibsondan December 17, 2021 19:50

alangenfeld requested changes Dec 17, 2021

View reviewed changes

dpeng817 added 6 commits January 7, 2022 11:28

Add OSS sensor backcompat tests

bd7c715

Bump latest tested backcompat version (0.12.8 backcompat is broken fo…

3723629

…r sensors)

Downbump to miminum working version

9464a61

bump version again

d3c97e0

Fix waiting for runs to finish

4aa3d93

Checking sensor health

350fe6d

dpeng817 force-pushed the sensor_backcompat_tests branch from cbe8a05 to 350fe6d Compare January 7, 2022 19:36

vercel bot temporarily deployed to Preview – dagit-storybook January 7, 2022 19:36 Inactive

vercel bot temporarily deployed to Preview – dagster January 7, 2022 19:37 Inactive

dpeng817 closed this Jan 26, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add OSS sensor backcompat tests #5889

Add OSS sensor backcompat tests #5889

dpeng817 commented Dec 7, 2021 •

edited

Loading

vercel bot commented Dec 7, 2021 •

edited

Loading

gibsondan commented Dec 9, 2021

dpeng817 commented Dec 9, 2021 •

edited

Loading

gibsondan left a comment

gibsondan Dec 15, 2021

alangenfeld left a comment

dpeng817 commented Jan 7, 2022

dpeng817 commented Jan 26, 2022

Add OSS sensor backcompat tests #5889

Add OSS sensor backcompat tests #5889

Conversation

dpeng817 commented Dec 7, 2021 • edited Loading

vercel bot commented Dec 7, 2021 • edited Loading

dagit-storybook – ./js_modules/dagit/packages/core

dagster – ./docs/next

gibsondan commented Dec 9, 2021

dpeng817 commented Dec 9, 2021 • edited Loading

gibsondan left a comment

Choose a reason for hiding this comment

gibsondan Dec 15, 2021

Choose a reason for hiding this comment

alangenfeld left a comment

Choose a reason for hiding this comment

dpeng817 commented Jan 7, 2022

dpeng817 commented Jan 26, 2022

dpeng817 commented Dec 7, 2021 •

edited

Loading

vercel bot commented Dec 7, 2021 •

edited

Loading

dpeng817 commented Dec 9, 2021 •

edited

Loading