Add Postgres Adapter #107

vermakhushboo · 2022-02-04T15:50:06Z

Work in progress

Edge cases in Appwrite but not in Utopia

Had to change the exception type to ‘Throwable’ instead of ‘PDOException’ for Appwrite as the exception wasn’t getting captured in createDocument and updateDocument methods
TODO:
- Add more tests that throw exceptions not caught by PDOException
- Find places where we throw the wrong errors
- Rethrow exceptions that Appwrite can understand
There are not enough tests in Utopia to verify that double-quotes were needed around {$attribute} in the find() method to handle reserved words
TODO:
- Find specific keywords that break this method
- Automate iteration to test for keywords
- Ex: https://www.drupal.org/docs/develop/coding-standards/list-of-sql-reserved-words#s-reserved-words
- Find more reserved words for different databases, example, https://en.wikipedia.org/wiki/SQL_reserved_words
Had to remove all alphanumeric values in getSQLCondition() method. Need to add better edge cases for full-text search in Utopia, it is missing many cases
TODO:
- Add more tests to cover similar scenarios in Utopia
- For example email ids etc
- Add tests specific to databases

Improvements needed in Appwrite for future adapters
TODO:

Hardcoded DB names in init.php, realtime.php, worker.php. Create generic interface for all adapters, allowing configurations using env variables
Add health check method to each adapter
Had to create a file PostgresSQLStatement.php and override Swoole methods, potentially move it to Utopia in the future

kevinjoshi46b · 2022-02-06T15:03:21Z

Hey @vermakhushboo ,
I have started working on adding postgres adapter a few days back. I had created a rfc for the same and had a discussion regarding me starting to work on it on appwrites discord server as well.
Are you working on it too?

vermakhushboo · 2022-02-07T15:40:49Z

Hey @vermakhushboo , I have started working on adding postgres adapter a few days back. I had created a rfc for the same and had a discussion regarding me starting to work on it on appwrites discord server as well. Are you working on it too?

Hey @KevinJ-hub, have you made considerable progress on the same? If yes, please take this forward, otherwise, I'd request you to pick something else.

kevinjoshi46b · 2022-02-07T15:49:58Z

Hey @vermakhushboo , I have started working on adding postgres adapter a few days back. I had created a rfc for the same and had a discussion regarding me starting to work on it on appwrites discord server as well. Are you working on it too?

Hey @KevinJ-hub, have you made considerable progress on the same? If yes, please take this forward, otherwise, I'd request you to pick something else.

Yes, I have started working on it from January. I had a discussion with the appwrite team on their discord server as well regarding me working on this and then on their approval, I started to work on it.

christyjacob4 · 2022-02-07T16:09:19Z

@KevinJ-hub Please create a PR with your changes. The postgres adapter has a high priority at the moment and we want to ensure you have enough time to pick it up. We're expecting to ship it in the coming week. Let us know if you would be able to commit to it 🙂

kevinjoshi46b · 2022-02-07T16:20:49Z

@KevinJ-hub Please create a PR with your changes. The postgres adapter has a high priority at the moment and we want to ensure you have enough time to pick it up. We're expecting to ship it in the coming week. Let us know if you would be able to commit to it slightly_smiling_face

@christyjacob4 Ohhh, I won't be able to commit on finishing it in a week since I am currently a bit caught up with college work. If thats the case then its fine @vermakhushboo you can please continue with postgres adapter I might pick something else and work on it. 👍

christyjacob4 · 2022-02-07T16:36:01Z

@KevinJ-hub Thank you so much for understanding! Really appreciate the honesty. If you need some guidance on issues you can pick up, please let me know. I'd be happy to assist.

kevinjoshi46b

Hey @vermakhushboo ,
The double quotes wrapped around role in read and write permission seem to be redundent over here since they do not get stored in the postgres DB. So I would like to request you to please check it once and remove them.

kevinjoshi46b · 2022-03-26T10:00:43Z

src/Database/Adapter/Postgres.php

+ VALUES ({$columns} :_uid, :_read, :_write)");
+
+ $read = array_map(fn($role) => '"'.$role.'"', $document->getRead());


The double quotes wrapped around role seem to be redundent

kevinjoshi46b · 2022-03-26T10:01:20Z

src/Database/Adapter/Postgres.php

+
+ $read = array_map(fn($role) => '"'.$role.'"', $document->getRead());
+ $write = array_map(fn($role) => '"'.$role.'"', $document->getWrite());


Same as above

kevinjoshi46b · 2022-03-26T10:01:39Z

src/Database/Adapter/Postgres.php

+ SET {$columns} _uid = :_uid, _read = :_read, _write = :_write WHERE _uid = :_uid");
+
+ $read = array_map(fn($role) => '"'.$role.'"', $document->getRead());


Same as above

kevinjoshi46b · 2022-03-26T10:01:44Z

src/Database/Adapter/Postgres.php

+
+ $read = array_map(fn($role) => '"'.$role.'"', $document->getRead());
+ $write = $read = array_map(fn($role) => '"'.$role.'"', $document->getWrite());


Same as above

…nversion

…at-postgress-adapter-eldad

…method

Merge latest postgres changes into original postgres PR

src/Database/Adapter/Postgres.php

abnegate

Looks really good! One small type change and some readability updates and I think we'll be good to go 😁

src/Database/Adapter/Postgres.php

Initial commit for Postgres Adapter

3ec40d1

vermakhushboo marked this pull request as draft February 4, 2022 15:50

vermakhushboo added 15 commits February 8, 2022 20:55

Creating schema and table

d838af9

testCreateExistsDelete() working

26de556

Resolved more errors

a316de7

More fixes

ee12fbe

Add more methods

369073d

Resolved attributes error and added deleteIndex method

19ebc2f

Refactor createCollection method

8abc0f1

Put createCollection into a transaction

3b6cd19

Fixed bool to int conversion

b88c335

Modify strings to array for permissions

7ccafbb

Modularise using encode & decode array methods

fe4c78c

Resolved getSQLCondition error

59efce8

All tests passing

e657a84

Merge main into branch & fix merge conflicts

235144c

Fixed errors and refactored code

567cc23

vermakhushboo marked this pull request as ready for review March 9, 2022 19:17

vermakhushboo requested a review from TorstenDittmann March 10, 2022 16:04

vermakhushboo added 2 commits March 18, 2022 22:25

Prepend collection name in index name

b7bfd8c

Typecast $internalId to string and wrap column name in quotes

0e41591

kevinjoshi46b suggested changes Mar 26, 2022

View reviewed changes

vermakhushboo added 2 commits April 1, 2022 00:42

Fixes for integration with Appwrite

e5fbd75

Fix merge conflicts

59d36d4

Khushboo Verma and others added 5 commits December 28, 2022 14:32

Clean up MariaDB file

46cc166

Replaced dynamic quotes with normal quotes for simplicity

46519a0

Minor fixes and improvements

4ef3d0a

Add SQL class and use it as parent class for MariaDB and Postgres

89aa916

Merge branch 'feat-postgresql-adapter' into feat-postgress-adapter-eldad

19c3a71

abnegate mentioned this pull request Jan 11, 2023

Feat postgress adapter (eldad) #195

Closed

Khushboo Verma and others added 12 commits January 11, 2023 16:58

Fixed renameIndex method and failing tests

9ce2ce0

Add updateAttribute method to Postgres

b080805

Fixed full-text search for postgres

8c030c9

Change TIMESTAMP to TIMESTAMP(3)

7f1cb6d

Added filters to updateAttribute method to handle 000 microseconds co…

2a43e7e

…nversion

Resolve merge conflicts

f728a02

Merge branch 'main' into feat-postgresql-adapter

9c1809c

Merge remote-tracking branch 'origin/feat-postgresql-adapter' into fe…

842811c

…at-postgress-adapter-eldad

Removed getAdapterRowLimit method from PostgresTest and fixed find() …

9cff2c0

…method

Fix count() method

847015a

Return false from getSupportForFulltextWildcardIndex() for postgres

b77e107

Merge pull request #225 from utopia-php/feat-postgress-adapter-eldad

013c60f

Merge latest postgres changes into original postgres PR

fogelito reviewed Jan 19, 2023

View reviewed changes

src/Database/Adapter/Postgres.php Show resolved Hide resolved

abnegate reviewed Jan 24, 2023

View reviewed changes

src/Database/Adapter/Postgres.php Outdated Show resolved Hide resolved

src/Database/Adapter/Postgres.php Outdated Show resolved Hide resolved

src/Database/Adapter/Postgres.php Outdated Show resolved Hide resolved

src/Database/Adapter/Postgres.php Outdated Show resolved Hide resolved

Khushboo Verma added 5 commits January 24, 2023 23:20

Merge branch 'main' into feat-postgresql-adapter

93db7f9

Resolved comments with minor fixes

3edba2c

Resolved comment

4a401a8

Added lower() method to unique index strings

84403eb

Fixed formatting

452b628

abnegate approved these changes Jan 25, 2023

View reviewed changes

stnguyen90 requested a review from fogelito January 25, 2023 23:32

fogelito approved these changes Jan 26, 2023

View reviewed changes

abnegate merged commit acb7f0a into main Jan 26, 2023

abnegate deleted the feat-postgresql-adapter branch November 23, 2023 01:50

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Postgres Adapter #107

Add Postgres Adapter #107

vermakhushboo commented Feb 4, 2022 •

edited

Loading

kevinjoshi46b commented Feb 6, 2022

vermakhushboo commented Feb 7, 2022

kevinjoshi46b commented Feb 7, 2022

christyjacob4 commented Feb 7, 2022

kevinjoshi46b commented Feb 7, 2022

christyjacob4 commented Feb 7, 2022

kevinjoshi46b left a comment •

edited

Loading

kevinjoshi46b Mar 26, 2022

kevinjoshi46b Mar 26, 2022

kevinjoshi46b Mar 26, 2022

kevinjoshi46b Mar 26, 2022

abnegate left a comment

		VALUES ({$columns} :_uid, :_read, :_write)");

		$read = array_map(fn($role) => '"'.$role.'"', $document->getRead());


		$read = array_map(fn($role) => '"'.$role.'"', $document->getRead());
		$write = array_map(fn($role) => '"'.$role.'"', $document->getWrite());

		SET {$columns} _uid = :_uid, _read = :_read, _write = :_write WHERE _uid = :_uid");

		$read = array_map(fn($role) => '"'.$role.'"', $document->getRead());


		$read = array_map(fn($role) => '"'.$role.'"', $document->getRead());
		$write = $read = array_map(fn($role) => '"'.$role.'"', $document->getWrite());

Add Postgres Adapter #107

Add Postgres Adapter #107

Conversation

vermakhushboo commented Feb 4, 2022 • edited Loading

Work in progress

kevinjoshi46b commented Feb 6, 2022

vermakhushboo commented Feb 7, 2022

kevinjoshi46b commented Feb 7, 2022

christyjacob4 commented Feb 7, 2022

kevinjoshi46b commented Feb 7, 2022

christyjacob4 commented Feb 7, 2022

kevinjoshi46b left a comment • edited Loading

Choose a reason for hiding this comment

kevinjoshi46b Mar 26, 2022

Choose a reason for hiding this comment

kevinjoshi46b Mar 26, 2022

Choose a reason for hiding this comment

kevinjoshi46b Mar 26, 2022

Choose a reason for hiding this comment

kevinjoshi46b Mar 26, 2022

Choose a reason for hiding this comment

abnegate left a comment

Choose a reason for hiding this comment

vermakhushboo commented Feb 4, 2022 •

edited

Loading

kevinjoshi46b left a comment •

edited

Loading