Added MySQL Support for DB options #3644

DevelopIdeas · 2022-08-14T15:51:29Z

For this like myself who already have systems running with a MySQL db we'd like the Frigate data in

netlify · 2022-08-14T15:51:32Z

✅ Deploy Preview for frigate-docs canceled.

Name	Link
🔨 Latest commit	`fe599a2`
🔍 Latest deploy log	https://app.netlify.com/sites/frigate-docs/deploys/62f9232a8b77eb0008a4be3c

NickM-27 · 2022-08-14T15:53:01Z

docker/Dockerfile.wheels

@@ -34,7 +34,8 @@ RUN pip3 wheel --wheel-dir=/wheels \
 matplotlib \
 click \
 setproctitle \
- peewee
+ peewee \
+ pymysql


For which part is pymysql needed?

pymysql is required by peewee when connecting to a MySQL DB, without this installed an Exception is raised

interesting, good to know, I figured they would have bundled it but that makes sense 👍

NickM-27 · 2022-08-14T15:55:34Z

frigate/app.py

@@ -33,6 +35,8 @@

 logger = logging.getLogger(__name__)

+class ReconnectMySQLDatabase(ReconnectMixin, MySQLDatabase):


With multiple db types being supported, probably makes sense to push this out to its own database.py

Sure, I can move the logic from frigate/app.py init_database to a database.py along with the import and recommit if you like?

I've done the refactor to database.py, do i need to create a new pull request to merge the changes?

They're all contained here so it's good as far as that goes 👍

Ah great! 👍

NickM-27 · 2022-08-14T15:56:10Z

frigate/app.py

- router.run()
-
- migrate_db.close()
+ if self.config.database.type == "sqlite":


Same as above, this can be moved to a database.py file and pass the config to it 👍

frigate/config.py

NickM-27 · 2022-08-14T16:00:43Z

frigate/config.py

+ default="frigate", title="Database password."
+ )
+ port: int = Field(
+ default=3306, title="Database password."


Suggested change

default=3306, title="Database password."

default=3306, title="Database server port."

NickM-27 · 2022-08-14T16:01:52Z

frigate/models.py



 class Event(Model):
 id = CharField(null=False, primary_key=True, max_length=30)
 label = CharField(index=True, max_length=20)
 camera = CharField(index=True, max_length=20)
 start_time = DateTimeField()
- end_time = DateTimeField()
+ end_time = DateTimeField(null=True)


Just curious, why is this needed?

I found when testing, pymysql doesnt set DEFAULT NULL without specifying this.
frigate/events.py tries to set this as None when starting an event.

NickM-27 · 2022-08-14T16:09:08Z

Thanks for your contribution!

Co-authored-by: Nicolas Mowen <[email protected]>

blakeblackshear · 2022-08-14T20:35:58Z

I'm not sure I want to add the overhead of compatibility with multiple database options.

DevelopIdeas · 2022-08-15T11:50:06Z

I'm not sure I want to add the overhead of compatibility with multiple database options.

I've got another commit relating to this.
Should I send it over or not bother if you're considering not adding support?

blakeblackshear · 2022-08-15T23:59:04Z

If was 100% plug and play, then I would consider it. There are a few places where I use custom sql queries, and I am not sure they will work with anything other than sqlite. I just frankly don't have the bandwidth to commit to testing multiple databases and maintaining any db specific workarounds for each release. If you were willing to commit to maintaining it and including a robust test suite alongside this, then maybe. I would likely just remove support as soon as it became a burden for me.

onedr0p · 2022-08-28T15:14:15Z

Personally I would rather see support using an ORM (like SQLAlchemy since this is python) rather than implementing a single database type. Postgres is a much more mature database over mariadb/mysql but feelings aside using an ORM would give people options to use either one in addition to sqlite.

I feel the pain of having to support this though and don't see much benefit over using sqlite unless you have hundreds of cameras, in which case you might want to spin up multiple instances of frigate anyways.

luisgreen · 2023-01-26T13:43:19Z

I'm not sure I want to add the overhead of compatibility with multiple database options.

Hi @blakeblackshear I've detected a race condition, I have 2 cameras that overlaps a zone, then they both detects the same event, sometimes only ONE gets recorded even if it where detected. I know they where detected because the MQTT event arrives my HA installation. Also, when a single camera gets two events, where two objeccts are too close, coincidentially when a person is strolling a dog... Sometimes it get the dog, sometimes it get the person.... But not both as it should. BUT both are detected for a tiny space of time. I'm not reading the full code, but it could be possible the database locking is preventing multiple transactions? If thats the case. Maybe you should consider moving towards a more robust DB.

Any volunteer to share the maintenance of a RDBMS with me?

NickM-27 · 2023-01-26T14:04:44Z

I have 2 cameras that overlaps a zone, then they both detects the same event, sometimes only ONE gets recorded even if it where detected. I know they where detected because the MQTT event arrives my HA installation. Also, when a single camera gets two events, where two objeccts are too close, coincidentially when a person is strolling a dog... Sometimes it get the dog, sometimes it get the person.... But not both as it should.

to be honest I doubt this has anything to do with the database. I've had scenarios like this plenty of times and I've never seen any issue with it recording all events to the DB that should have been recorded. Without seeing your config, the MQTT payloads, etc. there's no way to know this is related to the DB at all.

I'm not reading the full code, but it could be possible the database locking is preventing multiple transactions?

I'd suggest doing that because it seems the way you're assuming it works is incorrect.

blakeblackshear · 2023-01-27T03:16:56Z

@luisgreen what you are describing doesn't sound like a race condition or database lock issue to me. I don't think this closed PR has anything to do with what you are seeing.

If you want help troubleshooting what you are seeing, I suggest opening an issue and providing the requested information.

jangrewe · 2023-12-06T18:24:51Z

Could this be re-opened? I'd really love to see an ORM implementation, where i could just plug in any SQL URL and let the library handle the connection and queries, because let's be honest: SQLite sucks. ;-)

Added MySQL Support for DB options

119fa01

NickM-27 requested changes Aug 14, 2022

View reviewed changes

DevelopIdeas and others added 4 commits August 14, 2022 17:24

Update frigate/config.py

06c1146

Co-authored-by: Nicolas Mowen <[email protected]>

Refactored to database.py

0852e5b

Merge branch 'master' of https://github.com/DevelopIdeas/frigate

fe599a2

Fixed type

09983dd

NickM-27 mentioned this pull request Aug 27, 2022

Added MySQL Support for DB options #3643

Closed

github-actions bot added the stale label Sep 28, 2022

github-actions bot closed this Oct 2, 2022

NickM-27 mentioned this pull request Nov 3, 2022

[Support]: #4240

Closed

NickM-27 mentioned this pull request Jan 30, 2024

Any plans for add support for MariaDB/mySQL or PostgreSQL ? or how to improve SQLite ? #9496

Closed

NickM-27 mentioned this pull request May 20, 2024

Support other backend databases (Postgres?) #11460

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added MySQL Support for DB options #3644

Added MySQL Support for DB options #3644

DevelopIdeas commented Aug 14, 2022

netlify bot commented Aug 14, 2022 •

edited

Loading

NickM-27 Aug 14, 2022

DevelopIdeas Aug 14, 2022

NickM-27 Aug 14, 2022

NickM-27 Aug 14, 2022

DevelopIdeas Aug 14, 2022

DevelopIdeas Aug 14, 2022

NickM-27 Aug 14, 2022

DevelopIdeas Aug 14, 2022

NickM-27 Aug 14, 2022

NickM-27 Aug 14, 2022

NickM-27 Aug 14, 2022

DevelopIdeas Aug 14, 2022

NickM-27 commented Aug 14, 2022

blakeblackshear commented Aug 14, 2022

DevelopIdeas commented Aug 15, 2022

blakeblackshear commented Aug 15, 2022

onedr0p commented Aug 28, 2022 •

edited

Loading

luisgreen commented Jan 26, 2023

NickM-27 commented Jan 26, 2023 •

edited

Loading

blakeblackshear commented Jan 27, 2023

jangrewe commented Dec 6, 2023

		@@ -33,6 +35,8 @@

		logger = logging.getLogger(__name__)

		class ReconnectMySQLDatabase(ReconnectMixin, MySQLDatabase):

	default=3306, title="Database password."
	default=3306, title="Database server port."

Added MySQL Support for DB options #3644

Added MySQL Support for DB options #3644

Conversation

DevelopIdeas commented Aug 14, 2022

netlify bot commented Aug 14, 2022 • edited Loading

✅ Deploy Preview for frigate-docs canceled.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NickM-27 commented Aug 14, 2022

blakeblackshear commented Aug 14, 2022

DevelopIdeas commented Aug 15, 2022

blakeblackshear commented Aug 15, 2022

onedr0p commented Aug 28, 2022 • edited Loading

luisgreen commented Jan 26, 2023

NickM-27 commented Jan 26, 2023 • edited Loading

blakeblackshear commented Jan 27, 2023

jangrewe commented Dec 6, 2023

netlify bot commented Aug 14, 2022 •

edited

Loading

onedr0p commented Aug 28, 2022 •

edited

Loading

NickM-27 commented Jan 26, 2023 •

edited

Loading