[FR] integration tests which (a) mock API usage / (b) actually trigger API usage and create data stores, etc

[zeroasterisk]

Is your proposal related to a problem?

onboarding, wanting to adopt the tool or potentially contribute to the repo

Describe the solution you'd like

A part of getting started with the toolkit is proving that it works.

1. install dependancies
2. edit config and env files as needed
3. authenticate (via readme instruction)
4. run tests

Ideally the tests should be automated and consistent.

(a) Mock API Usage [default]

Running the test should not make changes in your account by default, but should simulate API usage and tell you what it would do.

This is simply proving that your code is working and mapping to API spec (as reflected in the mocks)

(b) Actual API Usage [opt-in]

You should be able to run the same set of tests which actually make changes in prod on your GCP project:

• Create Data Store test_scrapi_pdf_<uuid>
• Create Data Store test_scrapi_csv_<uuid>
• Create Data Store test_scrapi_blended_<uuid> (new)
• List Data Stores
• Assert Data Store Exists
• Add Docs
• Assert Docs Exist (minimal blob uploads from golden)
• [wait or decouple]
• Assert Docs Status as Processed
• Assert Data Store Valid Search API returns results & summary
• Assert Data Store Invalid Search API returns no results & deflection
• Upsert existing Docs
• Upsert new Docs
• Delete existing Docs
• [wait or decouple]
• Assert Docs Status as Processed
• Delete Data Store

Describe alternatives you've considered

More tutorial walkthroughs like Bot Building Series 101. If this is the chosen approach:

• Must extend for Data Stores and Docs with a similar pattern to above
• Must optimize for notebook reader (no execution should make sense)
• Must optimize for developer cycle - easy (re)run-all execution should work to verify (outer) development loop in prod

Additional context

Forked from #70 and CLI discussion.