project level secrets #3024

lohanidamodar · 2022-03-27T10:40:48Z

What does this PR do?

(Provide a description of what this PR does.)

Test Plan

(Write your test plan here. If you changed any code, please provide us with clear instructions on how you verified your changes work.)

Related PRs and Issues

(If this PR is related to any other PR or resolves any issue or related to any issue link all related PR and issues here.)

Have you read the Contributing Guidelines on issues?

(Write your answer here.)

…at-project-level-secrets

app/config/collections.php

eldadfux · 2022-05-22T11:47:01Z

app/config/collections.php

@@ -2668,15 +2690,15 @@
 'filters' => [],
 ],
 [
- '$id' => 'openSSLVersion',
+ '$id' => 'fileSecret',


Same, we should store a list of keys not a single one.

we are just using keys per file, so I don't think we need arrays here? Because if/when we change the key the file should already be encrypted using the new key.

eldadfux · 2022-05-22T11:47:35Z

app/controllers/api/account.php

@@ -1060,7 +1062,7 @@
 throw new Exception('No valid session found', 404, Exception::USER_SESSION_NOT_FOUND);
 }

- $jwt = new JWT(App::getEnv('_APP_OPENSSL_KEY_V1'), 'HS256', 900, 10); // Instantiate with key, algo, maxAge and leeway.
+ $jwt = new JWT($project->getId() == 'console' ? App::getEnv('_APP_OPENSSL_KEY_V1') : $project->getAttribute('jwtSecret'), 'HS256', 900, 10); // Instantiate with key, algo, maxAge and leeway.


Let's add a small explanation above this line.

Should we also keep older version to make sure we avoid any race conditions and failed validations during the switch? Not sure if we can ignore this scenario, wdyt?

I'm not sure? If we don't keep the old version, only thing is that all the existing JWT keys will be invalid right? Is that such a big problem? or am I missing something here?

As we already have console resource, I updated the console project to also include jwtSecrets attribute so that we don't need to check anything else here

eldadfux · 2022-05-22T11:49:20Z

app/controllers/api/functions.php

@@ -899,7 +899,7 @@
 }

 if(!$current->isEmpty()) {
- $jwtObj = new JWT(App::getEnv('_APP_OPENSSL_KEY_V1'), 'HS256', 900, 10); // Instantiate with key, algo, maxAge and leeway.
+ $jwtObj = new JWT($project->getId() == 'console' ? App::getEnv('_APP_OPENSSL_KEY_V1') : $project->getAttribute('jwtSecret'), 'HS256', 900, 10); // Instantiate with key, algo, maxAge and leeway.


Should we add JWT as a resource we initialize in one place?

as mentioned above, we can also try to cast the console attributes in a higher level to avoid code repetition.

we can do that I guess. I'll try that.

eldadfux · 2022-05-22T11:50:29Z

app/controllers/api/projects.php

@@ -98,6 +99,8 @@
 'keys' => null,
 'domains' => null,
 'auths' => $auths,
+ 'databaseSecrets' => [\uniqid() => \bin2hex(OpenSSL::randomPseudoBytes(128))],


Can we abstract this logic in the Auth library?

getting a new secret?

Shouldn't this be a part of OpenSSL instead of Auth library? I've put it on OpenSSL library. Do let me know WYT.

eldadfux · 2022-05-22T11:51:59Z

app/controllers/api/account.php

@@ -1060,7 +1062,7 @@
 throw new Exception('No valid session found', 404, Exception::USER_SESSION_NOT_FOUND);
 }

- $jwt = new JWT(App::getEnv('_APP_OPENSSL_KEY_V1'), 'HS256', 900, 10); // Instantiate with key, algo, maxAge and leeway.
+ $jwt = new JWT($project->getId() == 'console' ? App::getEnv('_APP_OPENSSL_KEY_V1') : $project->getAttribute('jwtSecret'), 'HS256', 900, 10); // Instantiate with key, algo, maxAge and leeway.


Can we set/cast these attributes (jwtSecret, databaseSecrets) globally on the console project object? Might be easier than forcing dev to remember adding this condition.

may be adding a JWT resource will be a good idea, I'll try that.

eldadfux · 2022-05-22T11:53:55Z

app/controllers/api/storage.php

@@ -508,9 +508,16 @@
 if (empty($data)) {
 $data = $deviceFiles->read($path);
 }
- $key = App::getEnv('_APP_OPENSSL_KEY_V1');
+
+ $fileSecret = \bin2hex(OpenSSL::randomPseudoBytes(128));


Same, can we abstract the secret generation?

eldadfux · 2022-05-22T11:55:11Z

composer.json

@@ -45,7 +45,7 @@
 "utopia-php/cache": "0.6.*",
 "utopia-php/cli": "0.12.*",
 "utopia-php/config": "0.2.*",
- "utopia-php/database": "0.15.*",
+ "utopia-php/database": "dev-feat-filters-upgrade as 0.15.4",


Reminder to release a version.

eldadfux · 2022-05-22T11:55:22Z

src/Appwrite/OpenSSL/OpenSSL.php

@@ -20,6 +20,7 @@ class OpenSSL
 */
 public static function encrypt($data, $method, $key, $options = 0, $iv = '', &$tag = null, $aad = '', $tag_length = 16)
 {
+ var_dump($data);


Co-authored-by: Eldad A. Fux <[email protected]>

…at-project-level-secrets

…at-sync-0.15

…at-pls-merge

…at-project-level-secrets

…ite/appwrite into feat-project-level-secrets

…at-project-level-secrets

stnguyen90 · 2023-10-05T17:29:41Z

Converting to draft due to merge conflicts.

lohanidamodar added 27 commits March 27, 2022 04:33

project level secrets data structure

8898cfa

generate keys during project creation

3d7f48e

set instance level filters to use database secrets

5f05264

encrypt files with storage secrets as well

7489ec4

remove comments

5750f09

using dev database

c4e3de8

fix missing code

7ca0696

remove unnecessary check

c09bf14

set proper instance level filters in realtime

89d4c82

filters in realtime

c923e2e

set instance filters in workers

e9b553f

update composer

d4897b2

fix error

2c00148

fix secret for version

33100df

convert bin to hex

057d566

make it work when project doesn't exist

feee2bf

increase secret column size

60ab3a8

getting empty project - during realtime test

77e9546

fix for console

6286bca

graceful when project not found

369a42c

encrypt jwt using jwt secret

967c751

update project level encryption decryption filter

87f8188

update secrets data type

aa92551

storage encryption new key per file

ea35445

Merge branch 'master' of https://github.com/appwrite/appwrite into fe…

3d43a97

…at-project-level-secrets

composer update

c4a7226

refactor encode and decode function

afa0206

lohanidamodar marked this pull request as ready for review May 8, 2022 06:41

fix functions and realtime

acad581

lohanidamodar requested a review from eldadfux May 8, 2022 08:04

lohanidamodar added 5 commits May 8, 2022 08:18

add json filter

6000177

refactor

44f81d9

open ssl version is no longer required

8718437

fix tests

c0a616c

fix tests on

f2482ab

eldadfux requested changes May 22, 2022

View reviewed changes

lohanidamodar and others added 7 commits May 23, 2022 07:26

Update app/config/collections.php

8f22a72

Co-authored-by: Eldad A. Fux <[email protected]>

Merge branch 'master' of https://github.com/appwrite/appwrite into fe…

43332fb

…at-project-level-secrets

fix attribute name

bcc0e0d

refactor jwt secrets

58d0005

secret generation extracted to open SSL

f403b82

fix realtime issues

7de5449

Merge branch '0.15.x' of https://github.com/appwrite/appwrite into fe…

77b38e3

…at-sync-0.15

lohanidamodar changed the base branch from master to 0.15.x June 3, 2022 06:00

lohanidamodar requested review from TorstenDittmann and eldadfux June 7, 2022 02:02

lohanidamodar changed the title ~~project level secrets data structure~~ project level secrets Jun 22, 2022

lohanidamodar added 6 commits June 29, 2022 05:52

Merge branch 'master' of https://github.com/appwrite/appwrite into fe…

7bd4e08

…at-pls-merge

fix linters

fab93f7

linter fixes

6248225

unused import

6c738b0

composer update

951a553

fix realtime

adc8be0

lohanidamodar changed the base branch from 0.15.x to master July 3, 2022 12:05

lohanidamodar added 4 commits July 14, 2022 04:52

Merge branch 'master' of https://github.com/appwrite/appwrite into fe…

3b62798

…at-project-level-secrets

Merge branch 'feat-project-level-secrets' of https://github.com/appwr…

d7a94cc

…ite/appwrite into feat-project-level-secrets

Merge branch 'master' of https://github.com/appwrite/appwrite into fe…

4cbef86

…at-project-level-secrets

Merge branch 'master' of https://github.com/appwrite/appwrite into fe…

0ead2a7

…at-project-level-secrets

stnguyen90 marked this pull request as draft October 5, 2023 17:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

project level secrets #3024

project level secrets #3024

lohanidamodar commented Mar 27, 2022

eldadfux May 22, 2022

lohanidamodar May 23, 2022

eldadfux May 22, 2022

eldadfux May 22, 2022

lohanidamodar May 23, 2022

lohanidamodar May 23, 2022

eldadfux May 22, 2022

eldadfux May 22, 2022

lohanidamodar May 23, 2022

eldadfux May 22, 2022

lohanidamodar May 23, 2022

lohanidamodar May 23, 2022

eldadfux May 22, 2022

lohanidamodar May 23, 2022

eldadfux May 22, 2022

lohanidamodar May 23, 2022

eldadfux May 22, 2022

eldadfux May 22, 2022

stnguyen90 commented Oct 5, 2023

project level secrets #3024

Are you sure you want to change the base?

project level secrets #3024

Conversation

lohanidamodar commented Mar 27, 2022

What does this PR do?

Test Plan

Related PRs and Issues

Have you read the Contributing Guidelines on issues?

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stnguyen90 commented Oct 5, 2023