Unable to Use '%' Sign in Database Password

[Trinkes]

Describe the bug
I've encountered an issue where I'm unable to use the '%' sign in my database password. This problem has been discussed in the Alembic repository (Issue #700), where it's noted that Alembic doesn't allow '%' signs in the password.

To work around this limitation, it's suggested to use urllib.parse.quote_plus to encode the password and replace the '%' with '%%' as described in this comment. However, this workaround results in an incorrect password in the SQLAlchemy context.

Steps to reproduce:

1. Attempt to use a database password containing the '%' sign in an Alembic migration or SQLAlchemy connection string.
2. Apply the suggested workaround of encoding the password with urllib.parse.quote_plus and replacing '%' with '%%'.
3. Note the behavior when attempting to use the password with both Alembic and SQLAlchemy.

Expected behavior:
The database password containing the '%' sign should be accepted and function properly in both Alembic and SQLAlchemy contexts.

Actual behavior:
Alembic does not allow the '%' sign in the password, and the suggested workaround results in an incorrect password in SQLAlchemy.

Traceback (most recent call last):
  File "/home/user/.local/lib/python3.10/site-packages/starlette/routing.py", line 734, in lifespan
    async with self.lifespan_context(app) as maybe_state:
      File "/usr/local/lib/python3.10/contextlib.py", line 199, in __aenter__
    return await anext(self.gen)
  File "/home/user/.local/lib/python3.10/site-packages/langflow/main.py", line 20, in lifespan
    initialize_services()
  File "/home/user/.local/lib/python3.10/site-packages/langflow/services/utils.py", line 217, in initialize_services
    raise exc
  File "/home/user/.local/lib/python3.10/site-packages/langflow/services/utils.py", line 214, in initialize_services
    initialize_database(fix_migration=fix_migration)
  File "/home/user/.local/lib/python3.10/site-packages/langflow/services/database/utils.py", line 52, in initialize_database
    raise exc
  File "/home/user/.local/lib/python3.10/site-packages/langflow/services/database/utils.py", line 32, in initialize_database
    database_service.run_migrations(fix=fix_migration)
  File "/home/user/.local/lib/python3.10/site-packages/langflow/services/database/service.py", line 130, in run_migrations
    alembic_cfg.set_main_option("sqlalchemy.url", self.database_url)
  File "/home/user/.local/lib/python3.10/site-packages/alembic/config.py", line 272, in set_main_option
    self.set_section_option(self.config_ini_section, name, value)
  File "/home/user/.local/lib/python3.10/site-packages/alembic/config.py", line 299, in set_section_option
    self.file_config.set(section, name, value)
  File "/usr/local/lib/python3.10/configparser.py", line 1206, in set
    super().set(section, option, value)
  File "/usr/local/lib/python3.10/configparser.py", line 896, in set
    value = self._interpolation.before_set(self, section, option,
                                                                  File "/usr/local/lib/python3.10/configparser.py", line 403, in before_set
    raise ValueError("invalid interpolation syntax in %r at "
ValueError: invalid interpolation syntax in 'postgresql:https://myuser:mypassword%s@langflow-postgres:5432/langflow' at position 30

Browser and Version

• Browser [N/A]
• Version [N/A]

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
trying to setup and use langfuse using docker.

[dosubot]

Hey there, @Trinkes! Great to see you around here again. Hope you're doing well.

Thank you for bringing this issue to our attention. It seems like you're encountering a problem with using a '%' sign in your database password due to Alembic's handling of '%' signs in the alembic.ini file.

To use a '%' sign in your database password for Langflow, you can follow these steps:

1. Encode the Password: Use urllib.parse.quote_plus to encode the password. This will convert special characters like '%' into URL-encoded format, which is '%25' for the '%' character.

2. Double the Percent Signs for Alembic: Since Alembic uses '%' as a special character for variable interpolation in the alembic.ini file, you need to escape any '%' characters by doubling them. However, since quote_plus already converts '%' into '%25', you'll need to replace '%' in the encoded password with '%%' to escape it properly for Alembic. This means '%25' becomes '%%25'.

3. Correctly Place the Password in the SQLAlchemy URL: Ensure the password is correctly placed within the SQLAlchemy database URL in the alembic.ini file.

Here's how you can do it in Python:

from urllib.parse import quote_plus

# Original password
password = "your_password_with_%_sign"

# Encode the password
encoded_password = quote_plus(password)

# Alembic requires '%%' to escape '%'
alembic_safe_password = encoded_password.replace('%', '%%')

# Now, use `alembic_safe_password` in your SQLAlchemy URL in the alembic.ini file

After encoding and escaping the password, you should manually insert it into the sqlalchemy.url in your alembic.ini file. For example, if your username is user and your database is hosted at localhost with a database name of langflow, the line in alembic.ini would look something like this:

sqlalchemy.url = postgresql:https://user:%%25EncodedPasswordHere@localhost/langflow

Replace %%25EncodedPasswordHere with the actual encoded and escaped password you generated with the script above. Remember, this example uses PostgreSQL as the database, but you should replace the URL scheme and path with those appropriate for your database (the original file uses SQLite, so adjust accordingly if you're using a different database).

Please note that the database password in the langflow codebase is handled through the DATABASE_URL setting. If a DATABASE_URL is not explicitly provided, the code first attempts to retrieve it from an environment variable named LANGFLOW_DATABASE_URL. If this environment variable is also not set, it defaults to using a SQLite database located at a path constructed from the CONFIG_DIR setting, specifically at <CONFIG_DIR>/langflow.db. If the SQLite database file ./langflow.db exists in the current directory and the new path does not already have a database file, it copies the existing database to the new location. If none of these conditions are met, it defaults to using ./langflow.db. This handling is implemented in the @validator("DATABASE_URL", pre=True) method of the Settings class. You can find more details in the source code.

I hope this helps! If you have any further questions or issues, please don't hesitate to ask.

Sources

• src/backend/langflow/alembic.ini
• src/backend/langflow/services/settings/base.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[Trinkes]

Isn't there a better way to do this other that editing the actual project's code?
if nothing else, can both urls (sqlachemy and alembic) can be exposed and set separately?

[ogabrielluiz]

Hey @Trinkes

I think I like the idea of having different variables for both as an optional override.
I'll look into that, probably today. If you think you could help or have any other input, please let me know.