This is my personal solution for photo management. It consists on the following pieces:
- Simple API server that receives picture uploads
- Simple processing queue that uploads pictures to S3, Flickr and GPhotos
- Basic web interface to browse S3 uploaded files
- Command line client to upload pictures
The idea is that I can upload my photos to a single endpoint and it will upload them on popular services so I can use their excellent browsing and sharing interfaces while I still maintain an original copy on an S3 bucket that I control.
The solution consists of 3 main parts:
- Upload API
- Processing queue
- Web interface
Exposes a single endpoint where to POST the photos. It will queue them for the processing queue to handle. Will temporarily store files in local file system.
Reads each file record from the queue and generates the thumbnails necessary for the web interface to display. Additionally it will upload the original file to S3 and to GPhotos and Flickr. Will delete the temporary local file when done.
A very basic interface to browse through the uploaded files. This is just to have a quick view on what's currently backed up.
You need to provide an environment variable SETTINGS
that should point to
a Yaml file containing the needed settings:
The Yaml file should have the following keys:
UPLOAD_FOLDER: <local filesystem directory for uploads temp files>
DB_FILE: <Sqlite db file, absolute path>
API_SECRET: <arbitraty string of your choice, shared with client>
S3_ACCESS_KEY: <AWS Access>
S3_SECRET_KEY: <AWS Secret>
S3_BUCKET: <Bucket name>
FLICKR_API_KEY: <API KEY>
FLICKR_API_SECRET: <API SECRET>
FLICKR_APP_TOKEN: <APP TOKEN>
FLICKR_APP_SECRET: <APP SECRET>
GPHOTOS_SECRET: <API SECRET>
GPHOTOS_CLIENT_ID: <CLIENT ID>
GPHOTOS_ACCESS_CODE: <PRISTINE ACCESS CODE>
DOMAIN: <Domain where photolog is hosted>
AUTH_ME: <Indieauth site for login>
SECRET_KEY: <Sessions secret key>
To obtain the needed credentials you will need to create an app type
"Desktop application" here https://www.flickr.com/services/api/keys/.
That will provide you with FLICKR_API_KEY
and FLICKR_API_SECRET
Then, open your python terminal
SETTINGS=settings.conf python
>>> import os
>>> from upload_api.settings import Settings
>>> settings = Settings.load(os.environ['SETTINGS'])
>>> import flickrapi
>>> api = flickrapi.FlickrAPI(settings.FLICKR_API_KEY, settings.FLICKR_API_SECRET)
>>> api.get_request_token(oauth_callback='oob')
>>> auth_url = api.auth_url(perms='write')
>>> print(auth_url)
# Paste this in browser and put in settings
https://www.flickr.com/services/oauth/authorize?oauth_token=xxxxxx&perms=write
>>> verifier_code = ###-###-###
api.get_access_token(verifier_code)
# Store the following settings
FLICKR_APP_TOKEN = api.flickr_oauth.oauth.client.resource_owner_key
FLICKR_APP_SECRET = api.flickr_oauth.oauth.client.resource_owner_secret
From your Google developer console, create a new project and new credentials for an "Oauth client ID".
That will provide with GPHOTOS_SECRET
and GPHOTOS_CLIENT_ID
Then, open your python terminal
SETTINGS=settings.conf python
>>> import os
>>> from upload_api.settings import Settings
>>> settings = Settings.load(os.environ['SETTINGS'])
>>> from upload_api.services.gphotos import get_access_code
>>> get_access_code(settings.GPHOTOS_CLIENT_ID)
Will print a url of the shape:
Paste that in your browser, accept and you will see the app access code. You
should put that in your GPHOTOS_ACCESS_CODE
setting.
You can use the main upload2photolog
command to get your pictures on your server.
It will handle many things.
The best way to have it ready is to create a file under ~/.photolog
with the
following:
host: <https://upload.host.com>
halt: <boolean>
secret: <same secret from API settings>
- host: Should point to the host where the API is installed. Should not include
any path from the API. If you installed it under
photos.domain.com
then you should havehttps://photos.domain.com
. - halt: Boolean, if true, the upload command will not break in case of a failed upload due bad connection, but will wait for user input to resume or cancel. This is very useful when performing huge uploads.
- secret: Same string you used on your API settings configuration, just to prevent bare HTTP calls from going through and avoid accidents in case you have many endpoints configured (my case during development)
Then, you can use it like so:
upload2photolog dir1 dir2 file1 file2 --tags 'tag1,tag2' --skip 'google,flickr'
And it will do its thing.
Upload a picture to the upload endpoint:
curl -X POST -F "[email protected]" -F "secret={{md5(secret)}}" https://localhost:5000/photos/
Use the tags
field to tag it:
curl -X POST -F "[email protected]" -F "secret={{md5(secret)}}" -F "tags=vegas,travel" https://localhost:5000/photos/
Or skip steps of the processing, in case you don't want your phone pictures uploaded to Gphotos again ("gphotos" or "flickr"):
curl -X POST -F "[email protected]" -F "secret={{md5(secret)}}" -F "skip=gphotos" https://localhost:5000/photos/
They will be added to the queue and processed sequentially. Will be available from your web interface.
Photolog is single user and uses Indieauth for
authentication. The site for authentication is configured under the AUTH_ME
setting, this site has to be Indieauth compatible.
Hit the /login/
url for signing in. Hit /logout/
for logging out.