Internal 184 log out redirect (#198)

* converted useInterval to timeMod, extracted dashboard styles, fixed useEffect bug of excessive calling

* broke styles into modules, formatting and addition of card info when not logged in

* removed logic to check if local for login, added else statement for rerender to display login message in uploadplugin and sidebar components

* fixed duplicate use state bug

* fixed grapl header styling for engagementview

* refactoring, fixed key errors for tables

* changed color of button

* black formatting

Release v0.2.0 (#186)

* Internal 9 fix workflow (#81)

* trigger on prereleases as well

* actions/checkout@v2 should checkout the correct branch anyway?

* bash

* fix tagged builds

* add graphql-endpoint to build scripts

* fix graphql endpoint Dockerfile

* docker cp

* zip arguments reversed >_<

* Add generic subgraph generator to build

* Build generic-subgraph-generator

* Add Dockerfile for generic-subgraph-generator

* Internal 9 fix workflow (#83)

* trigger on prereleases as well

* actions/checkout@v2 should checkout the correct branch anyway?

* bash

* fix tagged builds

* add graphql-endpoint to build scripts

* fix graphql endpoint Dockerfile

* docker cp

* zip arguments reversed >_<

* python lambda.zips are located at /home/grapl

* Internal 9 fix workflow (#84)

* trigger on prereleases as well

* actions/checkout@v2 should checkout the correct branch anyway?

* bash

* fix tagged builds

* add graphql-endpoint to build scripts

* fix graphql endpoint Dockerfile

* docker cp

* zip arguments reversed >_<

* python lambda.zips are located at /home/grapl

* use step outputs for artifact names

* New CDK.

* sneaky entry in parent .gitignore left the most important subdir out!

* Add grapl-notebook

* Restoring edge_ux - oops.

* Clean and rebuild of the edge_ux, as well as fixing graphql endpoint
string replacement.

* prefer arrow function.

* Bump service queue size from 1 to 10.

* Fixing perms granted by `allowReadWrite` to also grant write perms.

* Reduce console logs in schema.js

* Community docs (#85)

* initial attempt at community docs

* document branching strategy

* add link to Dobi issue

* Add script to download release zips by tag.

* Update to CDK 1.41.0.

* Update deploy_all.sh to reflect new stack name for UX.

* add grapl_analyzerlib to monorepo and fix rust containers

* only rename the binaries in the lambdas

* Add S3 encryption. Make fetch_Zips a bit more resilient.

* Move JWT to secretsmanager

* Remove unused import

* Publish grapl_graph_descriptions and grapl_analyzerlib to PyPI (#91)

* publish grapl_graph_descriptions and grapl_analyzerlib to PyPI

* fix a few mistakes

* remove some stuff that was erroneously brought in with grapl_analyzerlib

* run mypy in lint workflow

* fix a bug in the release workflow

* Fix release workflow (#92)

* publish grapl_graph_descriptions and grapl_analyzerlib to PyPI

* fix a few mistakes

* remove some stuff that was erroneously brought in with grapl_analyzerlib

* run mypy in lint workflow

* fix a bug in the release workflow

* fix workflow syntax

* whitespace

* Fix the grapl-release workflow (#93)

* publish grapl_graph_descriptions and grapl_analyzerlib to PyPI

* fix a few mistakes

* remove some stuff that was erroneously brought in with grapl_analyzerlib

* run mypy in lint workflow

* fix a bug in the release workflow

* fix workflow syntax

* whitespace

* what?

* whoops (#94)

* Internal 18 fix release workflow (#95)

* whoops

* >_<

* Fix pypa/gh-action-pypi-publish version (#96)

* whoops

* >_<

* use correct version for pypa/[email protected]

* Add rate limiting

* Remove unused import

* Valid emails for PyPI (#97)

* valid emails for PyPI

* update grapl-build and grapl-lint workflows to run on PR updates, run cargo update

* run lints and builds on the staging branch

* listen to synchronized events instead of edited

* set the working-directory before attempting cargo audit

* synchronize, not synchronized >_<

* update working-directory

* apparently working-directory does not work with uses

* make cargo audit actually work

* make cargo audit its own job s.t. it shows up as a separate check

* attempt to fix the grapl-cargo-audit job

* Internal 18 more release workflow fixes (#99)

* valid emails for PyPI

* update grapl-build and grapl-lint workflows to run on PR updates, run cargo update

* run lints and builds on the staging branch

* listen to synchronized events instead of edited

* set the working-directory before attempting cargo audit

* synchronize, not synchronized >_<

* update working-directory

* apparently working-directory does not work with uses

* make cargo audit actually work

* make cargo audit its own job s.t. it shows up as a separate check

* attempt to fix the grapl-cargo-audit job

* fix utility service builds

* add another depends_on (#100)

* Don't create a stack just for the secret. Use arn, not name.

* Fixing path join in fetch script.

* Remove engagement-graph from grapl_analyzerlib

* Bump versions for grapl_analyzerlib

* Use bump2version for python library releases (#101)

* version bumps for python libraries

* remove unreachable exceptions

* python3 (#102)

* Configure python correctly (#103)

* python3

* configure python correctly

* Check whether versions need to be bumped before building (#104)

* python3

* configure python correctly

* WIP so close

* exclude yanked packages

* some updates and fixes

* Remove .bumpversion.cfg and increment versions (#105)

* python3

* configure python correctly

* WIP so close

* exclude yanked packages

* some updates and fixes

* bump versions, remove .bumpversion.cfgs

* Complete removal of EngagementGraph

* Fix merge conflict

* Add caching to Dockerfile

* Black and rustfmt (#106)

* black

* rustfmt

* add a newline to trigger git checks

* cargo update

* run black once last time

* build in debian images instead of alpine (#108)

* bump  python library versions (#109)

* build in debian images instead of alpine

* bump python lib versions

* add PyPI checks to lint workflow

* - Adding version info as description to Lambdas.
- Naming cleanup.
- Pinning dgraph version. This should have been in a previous commit.

* DGraph - don't use standalone version in AWS.

* Fix rust builds (#110)

* build in debian images instead of alpine

* bump python lib versions

* add PyPI checks to lint workflow

* try to reduce peak disk usage for 10GB Github Actions limit

* Removing a file created from merge that shouldn't have made it in
previous commit.

* Move docs

* - Cleanup SageMaker Notebook name.

* - Remove encryption on UX bucket - this needs to be public.

* build in debian images instead of alpine (#108)

* bump  python library versions (#109)

* build in debian images instead of alpine

* bump python lib versions

* add PyPI checks to lint workflow

* Fix rust builds (#110)

* build in debian images instead of alpine

* bump python lib versions

* add PyPI checks to lint workflow

* try to reduce peak disk usage for 10GB Github Actions limit

* Renaming and syncing queue names in cdk with rest of the codebase.

* Fixing Python builds to work in AWS.

* Python and Rust formatting.

* Fix LGTM alert, unused import.

* Update sqs-lambda

* Add waiting for s3 and sqs

* Add aws_region to events

* Add ip_address to OutboundConnection

* - Adding version info as description to Lambdas.
- Naming cleanup.
- Pinning dgraph version. This should have been in a previous commit.

* DGraph - don't use standalone version in AWS.

* Removing a file created from merge that shouldn't have made it in
previous commit.

* - Cleanup SageMaker Notebook name.

* - Remove encryption on UX bucket - this needs to be public.

* Renaming and syncing queue names in cdk with rest of the codebase.

* Fixing Python builds to work in AWS.

* Python and Rust formatting.

* Fix LGTM alert, unused import.

* Remove engagement graph from model plugin deployer.

* Bump graph generator lib version

* Bump graph generator lib version

* Move grapl-config and grapl-graph-descriptions to crates.io

* Add license and description

* Add license and description

* Update Cargo.toml

* Update Cargo.toml

* Slip in a change to fix logging

* Update dependencies

* Syncing queue names.

* Fix unused import reported by LGTM.

* Attach reverse edges in DGraph (#114)

* implement reverse edges

* some bug fixes

* remove unused local

* remove some more engagement graph stuff

* add some debug logging

* Set up reverse edges, remove types

* fix logging

* put EngagementClient and EngagementView back in

* update for review comments

Co-authored-by: colin-grapl <[email protected]>
Co-authored-by: colin <[email protected]>

* Adding ability for generator lib s3 client to assume AWS STS role.

* internal-13-ui-updates (#118)

* fixed compiler warnings

* Start dashboard

* completed dashboard, page not found, extracted header, & routing

* Add file handler

* able to upload & delete plugins with UX changes

* reformatted using black

* lgtm changes

* Remove hanging reference to engagement_graph

* update version, fix encoding error

Co-authored-by: colin <[email protected]>

* Implement DGraph TTL cleanup job (#119)

* basic ttl job structure

* small fix

* small fixes

* code cleanup

* write batch query

* updates for review comments

* WIP but close now

* some updates from local testing

* updates from local testing

* make debug logs a little less spammy

* add grapl-dgraph-ttl to local grapl

* some more updates from local testing

* CDK for grapl-dgraph-ttl (#121)

* add --delay and --batch-size options to upload-sysmon-logs.py

* attempt to CDK

* add schedule rule

* updates for review comments

* address LGTM lint

* cdk README

* add extract-grapl-deployment-artifacts.sh

* WIP build is broken

* fix aws-cdk versions

* some updates from local testing

* WIP build still borked

* README update

* make dgraph-ttl a Construct instead of a NestedStack

* fix an issue with the handler reference

* some more updates from testing in sandbox

* more fixes

* normalize dgraph dns name handling (#126)

* Upgrade aws cdk to 1.46 to take advantage of fix for aws/aws-cdk#3463

* - Adding deploy name to support deploying multiple instances of Grapl to a
single AWS account.
- All Constructs now use an interface for constructor params beyond
scope and id. This essentially provides named params, making it clearer
what the arguments are for.

* Remove .env from .gitignore, env vars no longer used.

* Fix Grapl prefix from stackName issue.

* Fix EngagementUX Stack name.

* unused imports cleanup.

* Fix test code from template to new Grapl Stack params.

* Fixing some hard-coded names to take a prefix.

* Fix name for retry handler.

* Fixing hard-coded table names, which now take from env vars.

Also gonna mix up our naming convention. For fun.

* jkz colin

* Add asset_id_mappings table and remove node_id_retry_table.

* Add new env vars to docker-compose.yml

* Remove swp file that should not be there.

* Moving dynamodb table name fetches for node-identifier to grapl-config.

* Adding MG_ALPHAS to docker-compose for dgraph-ttl.

* Integrate cdk-watchful (#124)

* whitespace

* add cdk-watchful dep

* vendor cdk-watchful 0.5.1

* get vendored cdk-watchful to build

* add watchful to the Grapl stack

* some fixes

* remove cdk-watchful DynamoDB stuff because we use on-demand pricing

* remove unused import

* instrument engagement and graphql

* updates from local testing

* make everything a NestedStack

* internal-74 fix destroy_all.sh

* remove dangerous S3 stuff

* Updating CDK README to match changes from previous PR.

* Routing (#131)

* added paging for graphQL endpoint

* added front-end pagination to plugins

* graphQL paging

* fixed duplicate pagination bug for lenses

* added styling for paging

* formatting

* Revert extra lenses

* Styling

* fixed any types

* revisions

* refactored custom routing to use react-router-dom HashRouter

* remove redirectTo function

* added logic to check if logged in

* removed unneeded props

* Improve performance of plugin page and fix warnings

* Update sqs-lambda

* Update sqs-lambda

* Bump versions for tokio, tokio-compat, other dependencies

* Making the use of Watchful conditional on spec of email for watchful
alerts.

* Let's not forget to pass this new param to the Grapl Stack.

* Update graph-generator-lib

* Glob deps

* Properly pass in UX Bucket URL to graphql endpoint

* Fixing previous watcher commit.

* Move ux bucket to grapl-cdk-stack

* Fix import

* Fix .gitignore in CDK to not ignore the JS in our UX.

* modelPluginDeployer cdk hack

* Lowercase the origin in engagement edge

* Reformat

* Fix endpoint URL

* Use regional bucket domain with https

* Fix some more issues related to cors, env vars, etc

* Format engagement_edge

* Only use one client per service

* Reuse asset id db

* Formatting via Prettier. Adding .prettierrc.toml.

* Fixing bucket policies, which had a few issues:
  - ActionGetObject isn't a thing
  - resources for s3:GebObject action need path wildcard
  - extra perms in TODO note

* no sids.

* policy statement cleanup.

* Create the Lmabda execution role so we can name it. Also rename
publishesToBucket to writesToBucket.

* Adding explicit lambda execution roles with leigble names to ModelPluginDeployer
and DGraphTTL.

* Add description text to CDK stacks.

* Use dobi for builds (#146)

* WIP

* tag build images latest

* replace docker-compose build with dobi

* mark dobi-linux binary executable

* split builds into separate jobs

* small fix for release workflow

* RefactorBugfix

* Fixes a number of issues with localgrapl, and improves logging

* Temporary fix for cors in python services. grapl_analyzerlib plugin retriever fix. Notebook ACL fix.

* Fix acl for list of s3

* Improve error in dispatcher. Fix ACL

* Use addResources

* Fix a couple things:
  - pass new arg
  - change params to match scheme of others (extending Props)

* Don't unwrap a failed DGraph upsert

* Discard transaction on failure

* Remove commented out code

* Reformat python

* fix py formatting issues.

* Reformat python

* Reformat python

* Handle redeployment of plugins more gracefully

* Update Dockerfile to cd before zip

* It's a - not a _

* Fix underscores

* Give Engagement Notebook role a ligble name and a description.

* Fix a number of issues with the analyzer-executor

* Revert prefix

* Bump the grapl_analyzerlib VERSION

* Add paging to get_lenses (#125)

* added paging for graphQL endpoint

* added front-end pagination to plugins

* graphQL paging

* fixed duplicate pagination bug for lenses

* added styling for paging

* formatting

* Revert extra lenses

* Styling

* fixed any types

* revisions

* Reload page when login is successful and history changes. (#148)

* added location.reload() when history changes

* removed console.logs

* removed console.log

* Internal 135 error handling (#164)

* added validation

* added extra rows to plugin listing to prevent UI from jumping

* added yup package

* add back local_handlers for graph-merger and analyzer-dispatcher

* staging changes

* Tests in CI (#153)

* move assetdb and sessiondb tests under integration feature flag

* run rust tests in CI

* fix dobi configs for rust deploy images

* run python unit tests in builds

* run tests in js builds

* fix js tests

* WIP tests run but they fail

* WIP

* python integration tests pass!

* make rust integration tests run

* make rust integration tests work and wire tests into github workflows

* workflow improvements

* use cached build artifacts in integration tests

* remove bogus hypothesis examples

* Internal 37 fix workflows (#168)

* move assetdb and sessiondb tests under integration feature flag

* run rust tests in CI

* fix dobi configs for rust deploy images

* run python unit tests in builds

* run tests in js builds

* fix js tests

* WIP tests run but they fail

* WIP

* python integration tests pass!

* make rust integration tests run

* make rust integration tests work and wire tests into github workflows

* workflow improvements

* use cached build artifacts in integration tests

* remove bogus hypothesis examples

* fix failing release build

* fix build

* some fixes (#171)

* some fixes

* 😭

* Fix debug builds

* Remove test build step

* another release workflow fix (#175)

* Internal 82 local auth (#173)

* fixed bugs with local secretsmanager

* fixed dashboard buttons, bug fixes for local auth

* added UX_BUCKET_URL

* added missing parameter to GraphQLEndpoint for ux_bucket per Colin

* added dynamodb tables & service to graplprovision for local-grapl-user_auth

* removed links to documentation

* added welcome back

* debugging for local auth

* removed print statement

* check for print statement in graph-merger

* merge with staging

* cargo formatting

* python formatting

* run black for python formatting

* PR comment edits

* update documentation with localgrapl creds, python formatting

* reformatting

* fml (#177)

* Internal 37 moar workflow fixes (#178)

* fml

* another fix

* Bump sqs-lambda to 0.20.20

* Internal 172 canvas bug (#180)

* graphql query fixes

* extended DGraph TTL to 13 months

* remove console.log

* cargo update

* revert cargo update.

* Implement Github Actions caching (#170)

* WIP -- begin implementing gh actions caching

* WIP

* WIP

* run unit tests in dobi

* WIP workflows

* update workflows for new jobs

* add caching to build workflow

* fix workflow

* yaml syntax

* fix build workflow

* chicken or egg?

* copy pasta

* apparently artifacts are per-workflow

* remove upload/download

* fix build workflow

* another fix

Co-authored-by: inickles-grapl <[email protected]>

* Fix release workflow (#182)

* WIP -- begin implementing gh actions caching

* WIP

* WIP

* run unit tests in dobi

* WIP workflows

* update workflows for new jobs

* add caching to build workflow

* fix workflow

* yaml syntax

* fix build workflow

* chicken or egg?

* copy pasta

* apparently artifacts are per-workflow

* remove upload/download

* fix build workflow

* another fix

* release workflow updates

Co-authored-by: inickles-grapl <[email protected]>

* fix release flow (#185)

* fix release flow

* wat

* >_<

* Fix merge conflict (#187)

* Update grapl_analyzerlib

* Create CODE_OF_CONDUCT.md

* version bumps

* add newline for teh lulz

Co-authored-by: colin <[email protected]>
Co-authored-by: andrea-grapl <[email protected]>
Co-authored-by: colin-grapl <[email protected]>

* add check-pypi job to build workflow (#188)

* V0.2.0 docs update (#189)

* update CONTRIBUTING.md for v0.2.0 release

* README.md reformat and some updates

* add Grapl DFIR Slack invite link to CONTRIBUTING.md

* move check-pypi job to the lint workflow

* two more README.md updates

Co-authored-by: colin <[email protected]>
Co-authored-by: inickles-grapl <[email protected]>
Co-authored-by: colin-grapl <[email protected]>
Co-authored-by: inickles-grapl <[email protected]>
Co-authored-by: andrea-grapl <[email protected]>