Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/colang2 library migration #592

Open
wants to merge 30 commits into
base: develop
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
12795d2
feat(migration): implement migration script
Pouyanpi Jul 2, 2024
c530c7b
feat(cli): add `convert` command to cli for colang 1.0 to 2.x transla…
Pouyanpi Jul 2, 2024
78e83ee
feat(utils): add camelcase_to_snakecase and snake_to_camelcase utis
Pouyanpi Jul 2, 2024
ee7debe
feat(actions): Add action name normalization and registration check
Pouyanpi Jul 2, 2024
ff38a1b
feat(colang): enable context_updates in colang 2.x
Pouyanpi Jul 2, 2024
00ea1ce
Fix a minor bug in serialization
Pouyanpi Jul 2, 2024
aaef12c
feat(colang_parser): refactor colang parsing and add version detectio…
Pouyanpi Jul 2, 2024
f2c3bea
feat(guardrails.co): add global context vars in guardrails.co
Pouyanpi Jul 2, 2024
91abf3a
feat(core.co): add refuse to respond bot flow which is common in library
Pouyanpi Jul 2, 2024
171545c
feat(llmrails): add flow name validation for colang 2.x
Pouyanpi Jul 2, 2024
797bdbe
fix(library): fix format for colang 2 migration
Pouyanpi Jul 2, 2024
1a3d959
Migrate library to support colang 2.x
Pouyanpi Jul 2, 2024
46c83f4
docs(migration): add migration_guide.md
Pouyanpi Jul 2, 2024
159e831
fix(llmrails): fix issue with empty content in parse_colang_file() fu…
Pouyanpi Jul 2, 2024
4e56252
feat(config): add support for deprecated 'rails' configuration in Col…
Pouyanpi Jul 2, 2024
8052c95
fix(colang): fix indentation issue in _is_colang_v2 function
Pouyanpi Jul 2, 2024
6857b27
feat(migration): remove rails flows from config file after migration
Pouyanpi Jul 2, 2024
c55d3e4
fix(migration): fix action_name migration
Pouyanpi Jul 3, 2024
2832ac3
fix(migration): fix indentation issue
Pouyanpi Jul 3, 2024
7660a80
docs(cli): update cli.md
Pouyanpi Jul 3, 2024
70fddb9
feat(cli): refactor 'convert' command in CLI: Set 'use_active_decorat…
Pouyanpi Jul 3, 2024
33c018e
feat(tests): skip failing test
Pouyanpi Jul 4, 2024
1512d4e
feat(migration): refactor variable names and update syntax
Pouyanpi Jul 4, 2024
d3147b9
feat(migration): update regex for action name extraction
Pouyanpi Jul 4, 2024
dcf6a03
feat(migration): support ellipsis variable assignment
Pouyanpi Jul 4, 2024
9249587
fix(test_configs): correct missing quotation mark
Pouyanpi Jul 4, 2024
ccaa63d
feat(migration): Update syntax conversion and config handling
Pouyanpi Jul 4, 2024
97d9619
docs(migration): Enhance migration guide details
Pouyanpi Jul 4, 2024
eb3780e
feat(migration): Comment out rails flows instead of removing
Pouyanpi Jul 4, 2024
d2d2e2c
docs(migration): update return type in docstring of convert_co_file_s…
Pouyanpi Jul 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
docs(migration): Enhance migration guide details
  • Loading branch information
Pouyanpi committed Jul 5, 2024
commit 97d961950d43db83728f72a7ed8aa1873b35e595
56 changes: 36 additions & 20 deletions docs/user_guides/migration_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Migration Script

This script is designed to migrate colang modules and packages (co files) from Colang 1.0 format to Colang 2.0. It performs several transformations on the content of the files,i.e., syntax transformation, such as converting certain keywords and changing the case of certain identifiers.
This script is designed to migrate colang modules and packages (co files) from Colang 1.0 format to Colang 2.x. It performs several transformations on the content of the files, i.e., syntax transformation, such as converting certain keywords and changing the case of certain identifiers.

### How it Works

Expand All @@ -13,17 +13,45 @@ The script walks through the directory specified by the `path` argument and appl
- Converting `define bot` to `flow bot`
- Converting `define user` to `flow user`
- Converting `execute` to `await`
- Converting snake_case identifiers after `await` to CamelCase and append it with key word `Action`.
- Converting snake_case identifiers after `await` to CamelCase and append it with key word `Action` while preserving any arguments used.
- Converting `else when` to `or when`
- Converting `stop` to `abort`
- Converting quoted strings after `bot` to `bot say` or `or bot say`
- Converting quoted strings after `user` to `user said` or `or user said`
- Generate Anonymous flows, so when you are using a flow like below in colang 1
- Use `active` decorator for root flows (flows that need activation).
- Add `global` keyword to global variables.
- Converting `...` to corresponding syntax in colang 2
if you have a flow like below in colang 1
```colang
# some instruction in natural language
$var_name = ...
```
it translates it to colang 2 as below

```colang

#$name = await GenerateValueAction(instructions="some instruction in natural language")
```

or if we have a flow like below in colang 1
```colang
bot ...

# or
user ...
```
it translates it to colang 2 as below
```colang
UtteranceBotActionFinished()
# or
UtteranceUserActionFinished()

```

- If rails are defined in `config.yml` a `_rails.co` file is generated with the rails defined in it.

```co
```colang
define flow
user express greeting
bot express greeting
Expand All @@ -42,13 +70,17 @@ flow express_greeting

The script keeps track of the number of lines processed and the number of changes made in each file. It also counts the total number of files changed.

> Warning: The script modifies the original files. It is recommended to use version control to track the changes made by the script. It also enables you to see the differences between the original and modified files.

### Potential Issues and Weaknesses

- The script assumes that the input files are correctly formatted. If a file is not correctly formatted, the script may not work as expected.
- The script uses regular expressions to find and replace certain patterns in the text. If the input files contain text that matches these patterns but should not be replaced, the script may produce incorrect results.
- The script renames the original files and writes the transformed content to new files with the original names. Use version control to track the changes made by the script.
- The script does not handle errors that may occur during file reading and writing operations. If an error occurs, the script logs the error and continues with the next file.
- using characters like `-`, `+` and tokens like `or`, `and`, etc is not supported in flow definition, the migratin script does not handle this conversion due to handling. In case you have them try to fix them
- It is a better practice to define global variables at the begining of the flow however the migration script does not enforce this. We strongly advice you to do so manually.



### Using NeMo Guardrails CLI
Expand All @@ -63,20 +95,4 @@ The `convert` command has several options:

- `--verbose` or `--no-verbose`: If the migration should be verbose and output detailed logs. Default is `no-verbose`.
- `--validate` or `--no-validate`: If the migration should validate the output using Colang Parser. Default is `no-validate`.
- `--include-main-flow` or `--no-include-main-flow`: If the migration should add a main flow to the output. Default is `no-include-main-flow`.
- `--apply-active-decorator` or `--no-apply-active-decorator`: If the migration should use the active decorator. Default is `no-apply-active-decorator`.

Here are some examples of using these options:

```bash
nemoguardrails convert /path/to/directory --verbose
nemoguardrails convert /path/to/directory --validate
nemoguardrails convert /path/to/directory --include-main-flow
nemoguardrails convert /path/to/directory --apply-active-decorator
```

You can also combine multiple options:

```bash
nemoguardrails convert /path/to/directory --verbose --validate
```
- `--use-active-decorator` or `--no-use-active-decorator`: If the migration should use the active decorator. Default is `use-active-decorator`.