Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat initial push #4

Merged
merged 74 commits into from
Jun 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
e578aa6
Initial Push
PineappleIOnic Jan 5, 2023
2d467d0
Initial Push 2
PineappleIOnic Jan 17, 2023
8f769c3
Implement Supabase and improve Firebase and Appwrite adapters
PineappleIOnic Jan 18, 2023
649bffa
Add NHost and Tests
PineappleIOnic Jan 19, 2023
0770288
Continue code cleanup
PineappleIOnic Jan 24, 2023
d57cd4f
Add Appwrite Source Adapter
PineappleIOnic Jan 25, 2023
cb533f5
Implement Database Support
PineappleIOnic Feb 13, 2023
20615de
Add Subcollection Support for Firebase
PineappleIOnic Feb 16, 2023
0f6e82b
Update Psalm
PineappleIOnic Feb 16, 2023
1c0792c
Update composer.json
PineappleIOnic Feb 16, 2023
4c9bddb
Add getCurrent resource and populate Progress class
PineappleIOnic Feb 16, 2023
d707dc2
Stop using 8.2 features
PineappleIOnic Feb 17, 2023
a33d9e1
Fix User.php
PineappleIOnic Feb 17, 2023
fb05b47
Improve Checks function, start work on rewriting tests
PineappleIOnic Feb 23, 2023
347151f
Capitalise and normalise transfer resource names
PineappleIOnic Feb 24, 2023
a811b1d
Continue work on newer permission checks
PineappleIOnic Feb 24, 2023
8180478
Continue Work on validation
PineappleIOnic Feb 24, 2023
84b543d
Implement new report based check system
PineappleIOnic Feb 24, 2023
6260c98
Rollback Update
PineappleIOnic Feb 24, 2023
fae74e2
Use correct status codes
PineappleIOnic Feb 24, 2023
3949a5b
Various Fixes and Other things found while implementing API
PineappleIOnic Mar 22, 2023
3173861
First iteration
PineappleIOnic Mar 27, 2023
c042ac6
Improve algorithm for converting Indexes from Jake's feedback
PineappleIOnic Mar 27, 2023
ab75702
Finish Implementing Indexes for both NHost and Supabase
PineappleIOnic Mar 27, 2023
3138d03
Merge pull request #2 from utopia-php/feat-implement-indexes
PineappleIOnic Mar 27, 2023
568ba73
Finish Implementing Document support in both Nhost and Supabase
PineappleIOnic Mar 29, 2023
c17bb83
Implement Full Appwrite support
PineappleIOnic Apr 3, 2023
d32b002
Move playground out of tmp
PineappleIOnic Apr 3, 2023
020599b
Add and run formatter
PineappleIOnic Apr 3, 2023
579e024
Fix Playground Imports
PineappleIOnic Apr 3, 2023
e8d601a
Implement File Transfers for Appwrite
PineappleIOnic Apr 17, 2023
2318e6d
Begin implementing Relationships
PineappleIOnic Apr 17, 2023
fb2ad63
Merge branch 'feat-implement-files' into feat-initial-push
PineappleIOnic Apr 17, 2023
2373dc3
Delete .vscode directory
PineappleIOnic Apr 26, 2023
1745d1e
Continue Refactor work
PineappleIOnic Apr 26, 2023
9dcbdea
Update Target.php
PineappleIOnic Apr 26, 2023
bc47b31
Rework Appwrite File Transfer Client
PineappleIOnic Apr 26, 2023
04e2e08
Continue Refactor
PineappleIOnic May 10, 2023
417018c
Start implementing tests and change check with report
PineappleIOnic May 11, 2023
b8d27a2
Update progress tracking system
PineappleIOnic May 12, 2023
32a602b
Update Transfer.php
PineappleIOnic May 12, 2023
f56c320
Update Firebase.php
PineappleIOnic May 12, 2023
a8f4c12
Update Transfer.php
PineappleIOnic May 12, 2023
fec1470
Update Transfer.php
PineappleIOnic May 12, 2023
f1d41d3
add storage to firebase and change database system
PineappleIOnic May 17, 2023
ee8dd20
Run phpcbf and fix the linter issues
PineappleIOnic May 17, 2023
2b76b31
Merge File and FileData and remove psalm for pint
PineappleIOnic Jun 5, 2023
84c0be8
Rename ResourceCache to just Cache
PineappleIOnic Jun 5, 2023
8ee17a1
Continue work on PR updates
PineappleIOnic Jun 5, 2023
e45e1ae
Fix Visibility
PineappleIOnic Jun 5, 2023
53ee401
Update NHost.php
PineappleIOnic Jun 5, 2023
9f1c522
Continue Addressing PR issues
PineappleIOnic Jun 5, 2023
2e0cd4d
Continue work on tests
PineappleIOnic Jun 14, 2023
560bb93
Continue work on types
PineappleIOnic Jun 17, 2023
606e739
Merge branch 'feat-implement-tests' into feat-initial-push
PineappleIOnic Jun 17, 2023
6116d8d
Improve Null Username handling
PineappleIOnic Jun 17, 2023
c1861d2
Continue Work
PineappleIOnic Jun 19, 2023
75c4371
Update .env.example
PineappleIOnic Jun 21, 2023
e3af0cb
Update tests/e2e/adapters/MockSource.php
PineappleIOnic Jun 21, 2023
4aa03e9
Continue work on PR
PineappleIOnic Jun 21, 2023
b92c711
Update src/Transfer/Resources/Database/Attribute.php
PineappleIOnic Jun 21, 2023
209cb69
Update Attribute.php
PineappleIOnic Jun 21, 2023
af396c4
Merge branch 'feat-initial-push' of https://github.com/utopia-php/tra…
PineappleIOnic Jun 21, 2023
2119dfb
Continue work on attributes
PineappleIOnic Jun 21, 2023
28730fa
Update tests/Transfer/E2E/Sources/SourceCore.php
PineappleIOnic Jun 21, 2023
99ea59d
Update src/Transfer/Source.php
PineappleIOnic Jun 21, 2023
38d1235
Continue Work
PineappleIOnic Jun 21, 2023
16c6c50
Normalise const convention
PineappleIOnic Jun 21, 2023
15eea1c
Update hash vars
PineappleIOnic Jun 21, 2023
464ac5c
Fix more namessssss
PineappleIOnic Jun 21, 2023
9b73954
Fix Supabase
PineappleIOnic Jun 21, 2023
c1af62d
Update Supabase.php
PineappleIOnic Jun 21, 2023
9c8ccd8
Update Firebase.php
PineappleIOnic Jun 21, 2023
736a8f8
Fix Firebase
PineappleIOnic Jun 21, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
DESTINATION_APPWRITE_TEST_PROJECT=testProject
DESTINATION_APPWRITE_TEST_ENDPOINT=http:https://localhost/v1
DESTINATION_APPWRITE_TEST_KEY=xxxxxxxxxxxxxxxxxx

SOURCE_APPWRITE_TEST_PROJECT=testProject
SOURCE_APPWRITE_TEST_ENDPOINT=http:https://localhost/v1
SOURCE_APPWRITE_TEST_KEY=xxxxxxxxxxxxxxxxxx

FIREBASE_TEST_PROJECT=testProject
FIREBASE_TEST_ACCOUNT='{type: "service_account", ...}'

SUPABASE_TEST_ENDPOINT=https://xxxxxxxxxxxx.supabase.co
SUPABASE_TEST_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
SUPABASE_TEST_HOST=db.xxxxxxxxxxxxxxxx.supabase.co
SUPABASE_TEST_DATABASE=postgres
SUPABASE_TEST_USERNAME=postgres
SUPABASE_TEST_PASSWORD=xxxxxxxxxxxxxxxxxxxxx

NHOST_TEST_SUBDOMAIN=xxxxxxxxxxx
NHOST_TEST_REGION=eu-central-1
NHOST_TEST_SECRET=xxxxxxxxxxxxxxxxx
NHOST_TEST_DATABASE=xxxxxxxxxxxxxxx
NHOST_TEST_USERNAME=postgres
NHOST_TEST_PASSWORD=xxxxxxxxxxxxxxx


10 changes: 10 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
composer.lock
/vendor/
/.idea/
*.cache
.env
test-service-account.json
.DS_Store
localBackup/
.vscode/
.env.prod
76 changes: 76 additions & 0 deletions CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Contributor Covenant Code of Conduct

## Our Pledge

In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to make participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, sex characteristics, gender identity, expression,
level of experience, education, socio-economic status, nationality, personal
appearance, race, religion, or sexual identity and orientation.

## Our Standards

Examples of behavior that contributes to creating a positive environment
include:

- Using welcoming and inclusive language
- Being respectful of differing viewpoints and experiences
- Gracefully accepting constructive criticism
- Focusing on what is best for the community
- Showing empathy towards other community members

Examples of unacceptable behavior by participants include:

- The use of sexualized language or imagery and unwelcome sexual attention or
advances
- Trolling, insulting/derogatory comments, and personal or political attacks
- Public or private harassment
- Publishing others' private information, such as a physical or electronic
address, without explicit permission
- Other conduct which could reasonably be considered inappropriate in a
professional setting

## Our Responsibilities

Project maintainers are responsible for clarifying the standards of acceptable
behavior and are expected to take appropriate and fair corrective action in
response to any instances of unacceptable behavior.

Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.

## Scope

This Code of Conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community. Examples of
representing a project or community include using an official project e-mail
address, posting via an official social media account, or acting as an appointed
representative at an online or offline event. Representation of a project may be
further defined and clarified by project maintainers.

## Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting the project team at [email protected]. All
complaints will be reviewed and investigated and will result in a response that
is deemed necessary and appropriate to the circumstances. The project team is
obligated to maintain confidentiality with regard to the reporter of an incident.
Further details of specific enforcement policies may be posted separately.

Project maintainers who do not follow or enforce the Code of Conduct in good
faith may face temporary or permanent repercussions as determined by other
members of the project's leadership.

## Attribution

This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html

[homepage]: https://www.contributor-covenant.org

For answers to common questions about this code of conduct, see
https://www.contributor-covenant.org/faq
33 changes: 33 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
FROM postgres:alpine3.18 as supabase-db
COPY ./tests/Transfer/resources/supabase/backup.tar /docker-entrypoint-initdb.d/backup.tar
COPY ./tests/Transfer/resources/restore.sh /docker-entrypoint-initdb.d/restore.sh

FROM postgres:alpine3.18 as nhost-db
COPY ./tests/Transfer/resources/nhost/backup.tar /docker-entrypoint-initdb.d/backup.tar
COPY ./tests/Transfer/resources/restore.sh /docker-entrypoint-initdb.d/restore.sh

# Use my fork of mockoon while waiting for range headers to be merged
FROM node:14-alpine3.14 as mock-api
WORKDIR /app
RUN git clone https://github.com/PineappleIOnic/mockoon.git .
RUN npm run bootstrap
RUN npm run build:libs
RUN npm run build:cli
RUN mv ./packages/cli/dist/run /usr/local/bin/mockoon

FROM composer:2.0 as composer

WORKDIR /usr/local/src/

COPY composer.lock /usr/local/src/
COPY composer.json /usr/local/src/

RUN composer install --ignore-platform-reqs

FROM php:8.0-fpm-alpine3.14 as tests
RUN set -ex && apk --no-cache add postgresql-dev
RUN docker-php-ext-install pdo pdo_pgsql
COPY ./src /usr/local/src
COPY ./tests /usr/local/src/tests
COPY --from=composer /usr/local/src/vendor /usr/local/src/vendor
CMD php ./vendor/bin/phpunit
75 changes: 74 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,74 @@
# transfer
# Utopia Transfer

[![Build Status](https://travis-ci.com/utopia-php/transfer.svg?branch=main)](https://travis-ci.com/utopia-php/transfer)
![Total Downloads](https://img.shields.io/packagist/dt/utopia-php/transfer.svg)
[![Discord](https://img.shields.io/discord/564160730845151244?label=discord)](https://appwrite.io/discord)

Utopia Transfer is a simple and lite library to transfer and transform resources inbetween services. This library is aiming to be as simple and easy to learn and use. This library is maintained by the [Appwrite team](https://appwrite.io).

Although this library is part of the [Utopia Framework](https://github.com/utopia-php/framework) project it is dependency free and can be used as standalone with any other PHP project or framework.

## Getting Started

Install using composer:
```bash
composer require utopia-php/transfer
```

Init in your application:
```php
<?php

use Utopia\Transfer\Transfer;
use Utopia\Transfer\Sources\NHost;
use Utopia\Transfer\Destinations\Appwrite;

require_once __DIR__ . '/../../vendor/autoload.php';

// Initialize your Source
$source = new NHost('db.xxxxxxxxx.nhost.run', 'database-name', 'username', 'password');

// Initialize your Destination
$destination = new Appwrite('project-id', 'https://cloud.appwrite.io/v1', 'api-key');

// Initialize Transfer
$transfer = new Transfer($source, $destination);

// Transfer the resource groups you want
$transfer->run(
[
Transfer::GROUP_AUTH
], function ($status) {
echo $status['message'] . PHP_EOL;
}
);
```

## Supported Resources Chart

Sources:
| | Auth | Databases | Storage | Functions | Settings |
|----------|-------|-----------|-------|-----------|-----------|
| Appwrite | ✅ | ✅ | ✅ | ✅ | |
| Supabase | ✅ | ✅ | ✅ | | |
| NHost | ✅ | ✅ | ✅ | | |
| Firebase | ✅ | ✅ | ✅ | | |

Destinations:
| | Auth | Databases | Storage | Functions | Settings |
|----------|-------|-----------|-------|-----------|-----------|
| Appwrite | ✅ | ✅ | ✅ | ✅ | |
| Local | ✅ | ✅ | ✅ | ✅ | ✅ |

> **Warning**
> The Local destination should be used for testing purposes only. It is not recommended to use this destination in production or as a backup. The local destination is there to confirm that a source is working correctly and to test the transfer process with needing a target destination instance. This may change in the future however as the library matures.



## System Requirements

Utopia Transfer requires PHP 8.0 or later. We recommend using the latest PHP version whenever possible.

## Copyright and license

The MIT License (MIT) [http:https://www.opensource.org/licenses/mit-license.php](http:https://www.opensource.org/licenses/mit-license.php)
35 changes: 35 additions & 0 deletions composer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{
"name": "utopia-php/transfer",
"description": "A simple library to transfer resources between services.",
"type": "library",
"keywords": ["php", "framework", "upf", "utopia", "transfer"],
"license": "MIT",
"minimum-stability": "stable",
"authors": [{
"name": "Eldad Fux",
"email": "[email protected]"
},
{
"name": "Bradley Schofield",
"email": "[email protected]"
}
],
"autoload": {
"psr-4": {
"Utopia\\Transfer\\": "src/Transfer"
}
},
"autoload-dev": {
"psr-4": {"Utopia\\Tests\\": "tests/Transfer"}
},
"require": {
"php": ">=8.0",
"utopia-php/cli": "^0.13.0",
"appwrite/appwrite": "^8.0"
},
"require-dev": {
"phpunit/phpunit": "^9.3",
"vlucas/phpdotenv": "^5.5",
"laravel/pint": "^1.10"
}
}
47 changes: 47 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
version: '3'

services:
supabase-db:
build:
context: .
target: supabase-db
ports:
- "5432:5432"
networks:
- tests
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: postgres

nhost-db:
build:
context: .
target: nhost-db
networks:
- tests
ports:
- "5433:5432"
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: postgres

tests:
build:
context: .
target: tests
networks:
- tests
volumes:
- ./:/app
working_dir: /app
depends_on:
- supabase-db
- nhost-db
environment:
- NHOST_DB_URL=postgres:https://postgres:postgres@nhost-db:5432/postgres
- SUPABASE_DB_URL=postgres:https://postgres:postgres@supabase-db:5432/postgres

networks:
tests:
15 changes: 15 additions & 0 deletions phpcs.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
<?xml version="1.0"?>
<ruleset name="Appwrite Standard" namespace="Utopia">
<rule ref="PSR12"/>
<file>./src</file>
<file>./tests</file>
<ini name="memory_limit" value="4096M"/>
<!-- Ignore max line width -->
<rule ref="Generic.Files.LineLength">
<exclude-pattern>*</exclude-pattern>
</rule>
<!-- Allow Side Effects on root level of files -->
<rule ref="PSR1.Files.SideEffects.FoundWithSymbols">
<exclude-pattern>*</exclude-pattern>
</rule>
</ruleset>
16 changes: 16 additions & 0 deletions phpunit.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
<phpunit backupGlobals="false"
backupStaticAttributes="false"
bootstrap="vendor/autoload.php"
colors="true"
convertErrorsToExceptions="true"
convertNoticesToExceptions="true"
convertWarningsToExceptions="true"
processIsolation="false"
stopOnFailure="false"
>
<testsuites>
<testsuite name="E2E">
<directory>./tests/Transfer/E2E</directory>
</testsuite>
</testsuites>
</phpunit>
Loading