fixed backwards polarity on dones check #824
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adaickalavan
approved these changes
May 7, 2021
Gamenot
approved these changes
May 7, 2021
Gamenot
added a commit
that referenced
this pull request
Jul 2, 2021
* Envision near real time (#785) * Added frame_time in envision State, frame_time is the elapsed simulation time * Added "near real-time" and "uncapped" mode in Envision * add optimization to get frame_time from frame in server.py by using ijson instead of json loading the entire frame Co-authored-by: Stephen Hockema <[email protected]> * Zoo manager spawned as a subprocess (#814) * Envision fixes (#810) * Fixed race condition with seek flushing. * formatting * also limit stateQueue size while I'm at it. * More options for buffer overflow. * added comment * Quiet Envision logs (#820) * Quiet error received when initially unable to connect to Envision. * output a warning on the first failed connection only. * formatting * added to CHANGELOG * Quiet pybullet import msgs (#821) * quiet "pybullet build time: ..." messages on stdout when pybullet is imported. * added CHANGELOG entry * formatting * typo fix in CHANGELOG * fixed typo in filename * fixed backwards polarity on dones check (#824) * fix rounding cf timestp_sec to ensure we always keep enough digits (#825) * ULTRA Update callbacks (#818) * update episode callback * Apply suggestions from code review Co-authored-by: christianjans <[email protected]> Co-authored-by: christianjans <[email protected]> * Added a new pypi instructions file to update general user installation instructions * Ignore tune test in ULTRA's heavy CI tests (#811) * Make naming consistent for task 2's levels (#806) * Fixed typo in instructions * Cleaned up the instructions to only include the pip install instructions * Added further instructions about the modules involved in the package and how to use them to create environments and scenarios * updated CHANGELOG.md * updated README_pypi.md based on changes requested * fixed a few more minor typos * added command to remove all .coverageXX files after the tests commands exit * added a separate recipe to remove all the coverage files * minor bug fix * remove __init__.py files from scenarios and examples modules so that they are removed from python installation package or pip * Revert "remove __init__.py files from scenarios and examples modules so that they are removed from python installation package or pip" This reverts commit bdf80e0 * remove __init__.py file from scenarios module so that it is removed from python installation package or pip * rename the __init__.py file to argument_parser.py for examples folder * renamed README_pypi.md * Add social vehicles controlled by custom policy through bubbles (#793) * Add ability to include bubbles in scenarios * Add 'bubbles' to task configs * Add stops test * Add bubbles test * Refactor stops implementation in generate_scenarios.py * Refactor stops generation * Add type hints and docstrings * Remove stops from default bubble scenarios * Add __init__.py to tests/tune/ (#840) * Fix README typos in ULTRA (#764) * Fix typos in ultra/README.md * Fix typo in the citation * Fix headless argument behaviour in ULTRA (#737) * Fix headless argument in ULTRA scripts * Change headless flag to behave like SMARTS * Control number of episodes in ULTRA tests (rllib and train) (#739) * Make evaluations run in parallel to training * Add rollout_fragment_length and max_episode_steps to rllib * Add new flags to documentation * Fix flag in example * Ensure evaluation is not run in test_train.py * Check if log_dir doesn't exist in test_train.py * Fix typo in argument and set eval_episodes to 0 * Check if not exists for paths * Fix formatting * Update evaluate and train to evaluate per episode (#639) * Update evaluate and train to eval per episode * Fix tests and change normalization * Fix formatting * Record episode for train/eval at the same time * Train performance averages last eval_rate episodes * Add gap_mode and calculate generalization gap * Enable explore during eval for better comparison * Fix formatting * Fix averaging in record_episode * Only average scalars in record_episode * Check for scalar when calculating gap * Fix formatting * Fix bugs introduced by merge * Fix calculate_gap * Fix averaging over episodes * Fix formatting * Check eval_episodes is 0 before evaluation * Evaluate on train and test without exploration * Make train/test occur in one ray.get * Remove episode_count * Remove episode_count from tests * Remove averaging over training episodes * Restructure evaluation_check with eval_mode = False * Fix formatting * try split ci test method * Check for valid eval_episodes inside evaluation_check * Removed averaging feature of record_episode * Fix formatting Co-authored-by: Jenish Patel <[email protected]> * Merge with ultra-rllib-speedup-tests * Rename evaluation_task_id for training * Simplify argument to ray.wait(...) * Remove print statements * Fix defaults * Fix eval-rate in docs * Initial changes (#744) Co-authored-by: christianjans <[email protected]> Co-authored-by: Jenish Patel <[email protected]> Co-authored-by: christianjans <[email protected]> * Bump 0.4.15 -> 0.4.16 * Make format * Update missing version numbers * switched import statement to relative import * bug fix * made a new bin folder and moved gen_header.sh and header_test.sh to that folder * made a new etc folder and moved the third party folder into it * moved tools folder to etc folder * moved extras folder to etc * moved tools folder to examples folder * moved benchmark folder to etc folder * moved benchmark folder to cases folder and updated import statements * updated Makefile with proposed changes * moved install_deps.sh to etc/setup * updated Makefile based on changes proposed * moved docker files to etc/docker folder * fixed script paths in Makefile * fixed script paths in ci files * bug fix * bug fix * added recipe to remove coverageXXX and coverage.XXX files in Makefile * added README.pypi.md to etc/setup * fixed paths for dockerfile and made path changes in README.md * updated CHANGELOG.md * added __init__.py to scenarios and examples and removed them from pip installation in setup.py * updated based on proposed changes * revert back to absolute paths * added __init__.py * update CHANGELOG.md * new Imitation action space (#854) * first pass at new Imitation action space also fixed bug for non-dynamic vehicles not getting actions * Only do non-dynamic actions for agents not owned by a Provider * use actual mass from chassis instead of hardcoding * added hack to allow setting the initial vehicle speed * don't use Sumo here by default * added CHANGELOG entries * update from review: normalize by 4-wheel torque * ULTRA Improve high density traffic (#783) * experiment with different begin time functions * fixed map position in gif * added legends * WIP: experimenting with different combinations of traffic settings * save begin time settings * added more maps to high-density task, constrict stopwatcher begin time and t-intersection distribution improved from high-density traffic * formatted * clean up * final traffic settings saved * added mid-density to compare, modified high-density distribution * simplified the high density traffic to lessen congestion * more tuning * more tuning * added video analysis test * clean up * Apply suggestions from code review Co-authored-by: christianjans <[email protected]> * changes from code review * comment changes * slight mod to traffic distribution & relaxed stopwatcher begin times * Update begin_time_init_funcs.py Co-authored-by: christianjans <[email protected]> * remove debugging levels * small fix Co-authored-by: christianjans <[email protected]> * Remove extra comma in ULTRA citation (#849) * Remove extra comma in README citation * Use period instead of comma * revert back the file to prevent ci-test from failing for now * bug fix * reproducing bug * reproducing bug * bug fix * Fixed the bug by turning off the wrong_way event when vehicle goes into an intersection. Updated CHANGELOG.md. * Traffic History Dones (#827) * better handling of Dones for traffic history - detect vehicles dropping out of history - added ExitGoal for imitation learning agents * Updated CHANGELOG. * back out deprecation fix * allow for collisions to be reported in Events even if goal reached * Changed ExitGoal to TraverseGoal. Backed out semantic change to done Events. (feedback from dicussions) * type hints * updates from review & formatting * merge * Multiple replacement of history vehicles (#836) * First pass at multi-agent IL example * choose only overlapping vehicles to replace also, refactored TrafficHistory to clean up database queries * moved TraffiCHistory class to its own file * minor none check for replaced history vehicles * refactored overlapping_choice into TrafficHistory * updates from review * allow -k as a shortcut for --replacements-per-episode * formatting * added some type hints * more type hints * updates from review * updates from review * formatting * more type hints * fixed weird bug in last PR * updated based on changes proposed * updated based on changes proposed * updated based on changes proposed * removed the additional files to reproduce the bug * refactoring * reformatting sensors.py * reformatted sensors.py through black * updated CHANGELOG.md * added max-steps arg (#861) * Fix permission error. * moved the script etc/setup folder * ULTRA Simple level (#857) * added simple level * classified simple levels, position offsets Classified simple levels into v0, v1, v2. Move the position offsets to route distribution * small modification * pos_offset not mandatory * tweaked distribution flags & formatted * moved cross intersection types to test section in levels * bubbles flags added * renamed simple distribution, added a more general simple level * reinstated blocks distribution * Apply suggestions from code review Co-authored-by: christianjans <[email protected]> * moved debugging levels from task1 -> taskX Co-authored-by: christianjans <[email protected]> * update based on proposed changes * path fix * reformatted file using black * renamed etc folder to conf, moved README.pypi.md to utils/setup and supervisord.conf to examples * renamed cases to baselines * removed CONTRIBUTING.md from docs and updated the top level one * fixed CI test path * os path fix * reformatting file using black * Bugfix memory issues (#852) * Cap threadpoolexecutor virtual memory growth * Fix sumo memory growth * Make format * Apply suggestions * Update reset of sumo to every 50 * Add fixes to changelog * TODO to remove fix when memory growth fixed * revert back to 4 grpc connection workers * Make format * Bugfix remove unused `make` pview (#845) * Remove pview for unused .egg files * Update changelog * refactored repeating code to a class method * updated CHANGELOG.md * reformatted code * removed unit test file from another branch * ULTRA Update existing maps (#868) * update existing maps * formatted * changes from review * only observe cars for imitation learning (#874) * only observe cars for imitation learning data collection * added type hint * revert to absolute paths * prevent pybullet from updating during setup with python3.8 * updated CONTRIBUTING.md * resolve merge issue * ULTRA Add medium level (#869) * added medium level * adjusted traffic distr. in medium level * removed curvy_t intersection type * updated simple level moved existing simple level into taskX * cleaned up taskX config * changed level's total * Update ultra/ultra/scenarios/taskX/config.yaml Co-authored-by: christianjans <[email protected]> * updated simple level Co-authored-by: christianjans <[email protected]> * updated CHANGELOG.md * removed unwanted packages * updated requirements.txt * updated CHANGELOG.md * Changes based on reviews * Created a seperate setup.py for benchmark to remove opencv-python from SMARTS in general. * updated requirements.txt * updated requirements.txt * updated requirements.txt * examples and bug fixes for imitation learning (#878) * examples and bug fixes for imitation learning * added PR number to CHANGELOG * took out debugging hack * use BoxChassis (kinematics only, no physics) by default for Imitation action space * modified placeholder agent to replay from saved pkl files. +-get starting speed for mission, bug fixes * formatting * remove unnecessary dependencies * better rounding for history time query * do collision processing for non-dynamic action spaces too * fixed overzelous removal of provider type check + updates from review * added a few more type hints * reverted more over-zealousness re provider merging * fixed minor bug for when speed==0. * updates from review: better timestep_sec rounding * backed out Observation change + misc updates from review * re-compute traffic history speeds based on actual positions * Moved speed re-computation to genhistories to get out of step loop. - use next pos instead of prev pos in speed computation Fixed minor bug with obs_time hack. * don't check position on first step * enforce vehicle history limit to cars * fix for isclose() near 0 * move filtering by position to the end of genhistories (so all are used in smoothing stuff for edge effects) * change back to scalar acceleration for Imitation action space * formatting * use current and next pos to compute history speed * record and save the first observation (prior to step) too * fixed history replay 1-off timestep issue also added mission start adjustment hack for front bumper * ULTRA add `--durations` flag to ULTRA CI tests (#885) * added durations flag * test pypi ultra test * minor name fix * history agent vehicle sizing (#886) * allow traffic history agent vehicles' sizes to be set * simplified controller calculations (should not change them) * oops * use angular_velocity for position updates too * use current angular_velocity instead of next * reverted angular_velocity correction to position in controller also fixed bug to prevent stepping pybullet twice * Refactor Waypoints (#872) * waypoints/lanepoits refactor: first pass * incremental * move lanepoints into sumo_road_network * moved Waypoints (back) into MissionPlanner * added some type hints * minor cleanup * updated comment * added WaypointsCache and other minor performance optimizations * debugged waypoints cache * removed no-longer used method * updates from review * fixed typos * reduced lane-following lookahead to 10 * revert lane_following_controller lookahead reduction * added CHANGELOG entry * update per review: fixed hash for LinkedLanePoints * added more type hints per review * remove start/end offset from simple level (#891) * Proper shutdown of RemoteAgentBuffer after code crash (#822) * Removed atexit destroy in RemoteAgentBuffer. * fix internal holes in sumo road network * Updated CHANGELOG.md * code reformatting * code reformatting * comments * fixed all holes that were at extremely sharp turn (almost perpendicular). * traffic-history data speed and heading tweaks. (#896) * re-calculate speeds and headings based on position changes for both NGSIM and INTERACTION datasets * comment * ULTRA Update `docs/setup.md` (#903) * updated setup.md * remove manual black installation step * added install_deps.sh into ultra * Update ultra/docs/setup.md Co-authored-by: christianjans <[email protected]> * empty commit Co-authored-by: christianjans <[email protected]> * added an experiment utility file to replace context of supervisord.conf * reformatting * removed supervisord from dependencies and requirements * Ensured unique deque objects were used in framestack. (#894) * Updated README.md * updated CHANGELOG.md * updated requirements.txt * updated requirements.txt * updated requirements.txt * resolving merge conflict * resolving merge conflict * resolving merge conflict * path fixes * path fixes * Fix args count error caused by `websocket` (#906) * Fix vargs error sometimes caused by websocket * Change vargs to 2 additional defaults * Add fix to changelog * ignore options not defined in the command * reformatting * open new web browser tab automatically * reformatting * added scl zoo command to installl agents at the specified paths * Updated CHANGELOG.md * update tensorflow for Rl_Agent * Update getting_started.md (#925) * update tensorflow for Rl_Agent * added test file for rl-agent * updated the wheel file for rl-agent installation * updated based on reviews * added another option to change port * Small fix to docs/getting_started.md (#931) * cleanup * reformatting and more detail to error output * reformatting and more detail to error output * bump up the version number, update scenarios for rl-agent * updated wheel file for rl-agent * updated CHANGELOG.md * updated README.md and cleanup the environment before installing the agents. * updated README.md and cleanup the environment before installing the agents. * Add flag to disable mission shuffling when generating scenarios in ULTRA (#927) * Add option to generate scenarios without shuffling missions * Add comments to mission shuffling tests * Fix comments' flag names * Warning if port passed without starting envision. * Warning if port passed without starting envision. * remove argument parser (#935) * refactoring * revert changes * bug fix * bug fix * ULTRA Migrate from step-based to episode-based sampling (#932) * change agent_itr to episode.index * formatted * recording data episodically in tune.py and train.py * change recording_step inside evaluate_saved_model() * added episode indices test * Update ultra/tests/test_evaluate.py Co-authored-by: christianjans <[email protected]> * Update ultra/tests/test_evaluate.py Co-authored-by: christianjans <[email protected]> Co-authored-by: christianjans <[email protected]> * replace the panda3d.Quat.xform method to remove dependency from panda3D package * wrap the import statements and ShowBaseInstance class in try block and warn user if they have not installed the [render] dependencies. * make sure the CI tests install dependencies before rendering camera observations * remove panda3d from dependencies in setup.py * Updated CHANGELOG.md * ULTRA Run tests in forked subprocesses (#937) * forked train test * remove backslash * try forking evaluate test * small fix * small fix * small fix * comment added * update documentation about rendering and camera based observations * moved helper functions to utils/math * reformat the try block to raise RendererException with warning message. Comments. * rename the dependency name from render to camera-obs * Multi-agent FrameStack wrapper (#916) * Added multi-agent FrameStack wrapper and test. * cleaner way to make renderer optional * formatting * Refactor ULTRA adapter to support benchmarking (#871) * Initial ideas for refactored adapter * Define the reward and observation adapters * Finish basic adapter types in adapter.py * Get PPO baseline to run * Get SAC baseline to run * Get TD3 baseline to run * Get RLlib training to work * Get DQN baseline to run * Get BDQN baseline to run * Get ULTRA Tune to run * Pass ULTRA CI tests * Format the code * Move adapters to own directory * Remove original adapter and state preprocessors * Add discrete action Gym space * Implement default image observation adapter function * Simplify required_interface_fromtypes function * Initialize ultra_scores from enum type * Comment constants in default vector and reward * Change adapter CI test to use new adapters * Use seed from parameters for social vehicle config in RLlib * Add docstrings to adapters * Convert TODO regarding ULTRA score to NOTE in ULTRA environments * Fix small logic errors in default_observation_image_adapter * Apply small changes from review * Create deepcopy of observation before performing vector adaptation * Add initial documentation for default adapters * Add reason for sorting social vehicles by distance in adapters.md * Keep consistent line length in adapters.md * Add comments to image adapter constants * Edit docstring of image adapter function * Change image observation adapter to produce a 3D array * Test adapter's Gym space in adapter tests * Change gray-scale weighting to (0.1, 0.8, 0.1) * Fix 'enironment' typo and describe the_environment_reward in docs/adapters.md * Keep variable name of the rewards obtained from the environment consistent * User assertIn instead of assertTrue for testing containing in test_adapter.py * Add default info adapter to remove reward adapter's NeighborhoodVehicles dependency (#902) * Initial changes * Format the code * Remove ultra_scores from the ULTRA environment * Add info adapter documentation * Add info adapter test * Move shared adapter constants a common file * Make trimming of extra social vehiccles explicit * Replace RLlib environment info adaptation with default info adapter in agent * minor bug fix. Reformatting * Say which tests are run in ULTRA CI tests rather than use `--ignore` (#941) * Explicitly say which tests are run in heavy and light * Fix 'analysys' spelling mistake * ULTRA Minimalize `train.py` script (#904) * Initial ideas for refactored adapter * Define the reward and observation adapters * Finish basic adapter types in adapter.py * Get PPO baseline to run * Get SAC baseline to run * Get TD3 baseline to run * Get RLlib training to work * Get DQN baseline to run * Get BDQN baseline to run * Get ULTRA Tune to run * Pass ULTRA CI tests * Format the code * Move adapters to own directory * Remove original adapter and state preprocessors * Add discrete action Gym space * Implement default image observation adapter function * Simplify required_interface_fromtypes function * Initialize ultra_scores from enum type * Comment constants in default vector and reward * Change adapter CI test to use new adapters * Use seed from parameters for social vehicle config in RLlib * Add docstrings to adapters * Convert TODO regarding ULTRA score to NOTE in ULTRA environments * Fix small logic errors in default_observation_image_adapter * Apply small changes from review * Create deepcopy of observation before performing vector adaptation * Add initial documentation for default adapters * save work * moved parsing outside of main * made build_agent() public * empty commit * added max_steps args * empty commit * small fix * changes from code review * small fix * bypass hard coded file name * clean up Co-authored-by: christianjans <[email protected]> * Add trajectory interpolation provider. * Update scenario building documentation and arguments in ULTRA (#944) * Fix scl scenario build command in docs * added default value to pool_dir arg * make path relative to base ultra dir * update scenario README * add logic clause to allow build_scenarios() function calls to not mandate using pool_dir argument * Change pool_dir initialization in build_scenarios Co-authored-by: Jenish Patel <[email protected]> * Add frame stacking to ULTRA (#948) * Initial changes * Add initial environment documentation * Fix ULTRA environment and adapter test to pass tests * Clean up the ULTRA environment and environment test * Add note to RLlib training about SMARTS observations adaptation * Update ULTRA environment documentation * update MANIFEST.in (#956) * fix multi-display envision * prevent envision to reconnect to a simulation every 3 second * prevent envision to reconnect to a simulation every 3 second * prevent envision to reconnect to a simulation every 3 second. bug fixes. * Updated CHANGELOG.md * Build and Update the envision distribution based on changes. * Bug fixes. Major reformatting for .js files. * build and update map * fix envision to check for a new simulation every 3 seconds and not redirect when switched to a multi-instance display. * build and update the distribution. Reformatting. * typo fix * changes based on reviews * fork all light base tests and observe duration (#967) * ULTRA Evaluate specific models (#915) * refactor existing evaluate_saved_models * fixed bug * updated evaluation process, fixed tune test issue * Update getting_started.md * updated getting_started.md documentation * clean up * added exceptions * small fix Co-authored-by: christianjans <[email protected]> Co-authored-by: christianjans <[email protected]> Co-authored-by: christianjans <[email protected]> * ULTRA baselines with new adapters (#958) * Initial ideas for refactored adapter * Define the reward and observation adapters * Finish basic adapter types in adapter.py * Get PPO baseline to run * Get SAC baseline to run * Get TD3 baseline to run * Get RLlib training to work * Get DQN baseline to run * Get BDQN baseline to run * Get ULTRA Tune to run * Pass ULTRA CI tests * Format the code * Move adapters to own directory * Remove original adapter and state preprocessors * Add discrete action Gym space * Implement default image observation adapter function * Simplify required_interface_fromtypes function * Initialize ultra_scores from enum type * Comment constants in default vector and reward * Change adapter CI test to use new adapters * Use seed from parameters for social vehicle config in RLlib * Add docstrings to adapters * Convert TODO regarding ULTRA score to NOTE in ULTRA environments * Fix small logic errors in default_observation_image_adapter * Apply small changes from review * Create deepcopy of observation before performing vector adaptation * Add initial documentation for default adapters * Add reason for sorting social vehicles by distance in adapters.md * Keep consistent line length in adapters.md * Add comments to image adapter constants * Edit docstring of image adapter function * Change image observation adapter to produce a 3D array * Initial changes * Fix bug in image replay buffer dataset * Add initial replay buffer test * Clean up replay buffer test * Make CNNs exclusively process images * Remove BDQN's explore.py and network.py * Remove extra import of DQNWithSocialEncoder * ULTRA Sanity check on baselines (#965) * added test to train all baselines * formatted * added evaluation to baseline tests * fix agent interface numbering (#978) Co-authored-by: Rutvik Gupta <[email protected]> * fixed genhistories bug introduced in PR #896. * Graceful shutdown of zoo when daemon process terminates abruptly (#924) * Update version number Co-authored-by: Jeffer Jingfei Peng <[email protected]> Co-authored-by: Stephen Hockema <[email protected]> Co-authored-by: adai <[email protected]> Co-authored-by: sah-huawei <[email protected]> Co-authored-by: JenishPatel99 <[email protected]> Co-authored-by: christianjans <[email protected]> Co-authored-by: Rutvik Gupta <[email protected]> Co-authored-by: Rutvik Gupta <[email protected]> Co-authored-by: Alex Lewandowski <[email protected]> Co-authored-by: christianjans <[email protected]> Co-authored-by: Jenish Patel <[email protected]> Co-authored-by: Adaickalavan Meiyappan 84166141 <[email protected]> Co-authored-by: guptarut <https://markus.cs.teach.edu/git/csc209-2020-01/guptarut> Co-authored-by: Valax Kong <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I recently changed the
donescheck in this script to not crash when theagent_idis not found, but I got the default value backwards. If theagent_idisn't found, it's not still running, and hence it's "done".(pointed out by @zbzhu99)