UnloadPrompts: Add a config toggle for whether or not tgis prompt art…

…ifacts should be unloaded This allows for the prompt cache to be purged when it's something that the runtime "owns" and for the runtime to not purge the cache if the cache is the same as local_models_dir (auto-load prompts). caikit#180 Signed-off-by: Gabe Goodhart <[email protected]>
flaviabeo · Sep 8, 2023 · a7e7811 · a7e7811
1 parent c16c33a
commit a7e7811
Showing 1 changed file with 3 additions and 0 deletions.
diff --git a/caikit_nlp/config/config.yml b/caikit_nlp/config/config.yml
@@ -21,5 +21,8 @@ torch_dtype: float32
 # Path of folder that will contain all the source prompts
 source_prompt_base: ""
 
+# Whether or not to purge TGIS prompts on model deletion
+unload_tgis_prompt_artifacts: false
+
 runtime:
  library: caikit_nlp