Automatically share data between popular services you use on the web. And instead of giving your credentials to them, become the owner of yours !
For example a new RSS item is published, django-trigger-happy will be able to automatically create a note on your Evernote account or create a bookmark to your own Readability or Pocket account and so on
The goal of this project is to be independant from any other solution like IFTTT, CloudWork or others.
Thus you could host your own solution and manage your own triggers without depending any non-free solution.
With this project you can host triggers for you.
All you need is to have a hosting provider (or simply your own server ;) ) who permits to use a manager of tasks like "cron" and, of course Python.
- Python 3.4.x
- Django >= 1.8
- arrow == 0.5.4
- django-formtools == 1.0
- django-js-reverse == 0.5.1
- libtidy-dev >= 0.99
The latest libtidy-dev should be installed with your operating system package manager, not from pip. On a Ubuntu system:
apt-get install libtidy-dev
for celery
- Celery == 3.1.18
for evernote support
for pocket support
- pocket == 0.3.5
for readability support
- readability == 1.0.0
for rss support
- feedparser == 5.1.3
for search engine
- django-haystack == 2.3.1
for trello support
- trello == 0.4.3
for twitter support
- twython == 3.2.0
for redis support
- django-redis == 4.1.0
- django-redisboard == 1.2.0
To get the project up and running, from your virtualenv, do:
git clone https://github.com/foxmask/django-th.git
then continue by installing :
cd django-th
python setup install
cd ..
pip install -r requirements-evernote.txt
in 2 steps :
pip install django-th
and you will have to finish by
pip install -r https://raw.githubusercontent.com/foxmask/django-th/master/requirements-evernote.txt
this is because Evernote SDK for Python 3 is not yet available on pypi
As usual you will setup the database parameters.
Important parts are the settings of the available services :
add the module django_th to the INSTALLED_APPS
INSTALLED_APPS = (
...
'formtools',
'django_js_reverse',
'redisboard',
'django_th',
'th_rss',
'th_pocket',
'th_readability',
'evernote',
'th_evernote',
'th_twitter',
'th_holidays',
'th_trello',
'haystack', # mandatory if you plan to use th_search
'th_search', # then follow instructions from https://django-haystack.readthedocs.org/
)
TH_SERVICES is a list of the supported services
TH_SERVICES = (
# comment the line to disable the service you dont want
'th_rss.my_rss.ServiceRss',
'th_pocket.my_pocket.ServicePocket',
'th_evernote.my_evernote.ServiceEvernote',
'th_readability.my_readability.ServiceReadability',
'th_trello.my_trello.ServiceTrello',
'th_twitter.my_twitter.ServiceTwitter',
)
TH_EVERNOTE is the settings you will need to be able to add/read data in/from Evernote.
To be able to use Evernote see official FAQ :
TH_EVERNOTE = {
'sandbox': True, #set to False in production - to be able to use it with trigger happy of course ;)
'consumer_key': 'abcdefghijklmnopqrstuvwxyz',
'consumer_secret': 'abcdefghijklmnopqrstuvwxyz',
}
TH_POCKET is the settings you will need to be able to add/read data in/from Pocket.
To be able to use Pocket :
- you will need to grad the pocket consumer key by creating a new application with the rights access as below
- then copy the "consumer key" of your application to the settings.py
TH_POCKET = {
'consumer_key': 'abcdefghijklmnopqrstuvwxyz',
}
TH_READABILITY is the settings you will need, to be able to add/read data in/from readability Service.
To be able to use readability :
- you will need to grad the readability keys
- create a new application at readability website, then
- copy the "keys & secret" of your application to the settings.py
TH_READABILITY = {
'consumer_key': 'abcdefghijklmnopqrstuvwxyz',
'consumer_secret': 'abcdefghijklmnopqrstuvwxyz',
}
TH_TRELLO is the settings you will need to be able to add/read data in/from Trello.
To be able to use Trello:
- you will need to create an account at https://trello.com/docs/
- then create an application and adding to the URL request "scope=read,write"
TH_TRELLO = {
'consumer_key': 'abcdefghijklmnopqrstuvwxyz',
'consumer_secret': 'abcdefghijklmnopqrstuvwxyz',
}
TH_TWITTER is the settings you will need to be able to add/read data in/from Twitter.
To be able to use Twitter:
- you will need to create an account at https://apps.twitter.com/
- then create an application
- then on the Application Settings tab set the rights to "read and write permission"
- then on Keys tab copy the information and fill the settings.py with them
TH_TWITTER = {
'consumer_key': 'abcdefghijklmnopqrstuvwxyz',
'consumer_secret': 'abcdefghijklmnopqrstuvwxyz',
}
For each TriggerHappy component, define one cache like below
CACHES = {
# Evernote Cache
'th_evernote':
{
'TIMEOUT': 500,
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": "127.0.0.1:6379",
"OPTIONS": {
"DB": 1,
"CLIENT_CLASS": "django_redis.client.DefaultClient",
}
},
# Pocket Cache
'th_pocket':
{
'TIMEOUT': 500,
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": "127.0.0.1:6379",
"OPTIONS": {
"DB": 2,
"CLIENT_CLASS": "django_redis.client.DefaultClient",
}
},
# RSS Cache
'th_rss':
{
'TIMEOUT': 500,
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": "127.0.0.1:6379",
"OPTIONS": {
"DB": 3,
"CLIENT_CLASS": "django_redis.client.DefaultClient",
}
},
# Readability
'th_readability':
{
'TIMEOUT': 500,
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": "127.0.0.1:6379",
"OPTIONS": {
"DB": 4,
"CLIENT_CLASS": "django_redis.client.DefaultClient",
}
},
# Trello Cache
'th_trello':
{
'TIMEOUT': 500,
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": "127.0.0.1:6379",
"OPTIONS": {
"DB": 5,
"CLIENT_CLASS": "django_redis.client.DefaultClient",
}
},
# Twitter Cache
'th_twitter':
{
'TIMEOUT': 500,
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": "127.0.0.1:6379",
"OPTIONS": {
"DB": 6,
"CLIENT_CLASS": "django_redis.client.DefaultClient",
}
},
}
Celery will handle tasks itself to populate the cache from provider services and then exploit it to publish the data to the expected consumer services
- From Settings
Define the broker then the scheduler
from celery.schedules import crontab
BROKER_URL = 'redis:https://localhost:6379/0'
CELERYBEAT_SCHEDULE = {
'read-data': {
'task': 'django_th.tasks.read_data',
'schedule': crontab(minute='27,54'),
},
'publish-data': {
'task': 'django_th.tasks.publish_data',
'schedule': crontab(minute='59'),
},
}
- From SUPERVISORD
[program:django_th_worker]
user = foxmask
directory=/home/projects/trigger-happy/th
command=/home/projects/trigger-happy/bin/celery -A django_th worker --autoscale=10,3 -l info
autostart=true
autorestart=true
redirect_stderr=true
stdout_logfile=/home/projects/trigger-happy/logs/trigger-happy.log
stderr_logfile=/home/projects/trigger-happy/logs/trigger-happy-err.log
[program:django_th_beat]
user = foxmask
directory=/home/projects/trigger-happy/th
command=/home/projects/trigger-happy/bin/celery -A django_th beat -l info
autostart=true
autorestart=true
redirect_stderr=true
stdout_logfile=/home/projects/trigger-happy/logs/trigger-happy.log
stderr_logfile=/home/projects/trigger-happy/logs/trigger-happy-err.log
# REDISBOARD
REDISBOARD_DETAIL_FILTERS = ['.*']
if you plan to use the search feature, put the engine of your choice, for example :
# needed to th_search and haystack
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine',
'URL': 'https://127.0.0.1:9200/',
'INDEX_NAME': 'haystack',
},
}
from django.conf.urls import patterns, include, url
from django.contrib import admin
urlpatterns = patterns('',
# Examples:
# url(r'^$', 'th.views.home', name='home'),
# url(r'^blog/', include('blog.urls')),
url(r'^admin/', include(admin.site.urls)),
url(r'', include('django_th.urls')),
)
Once the settings is done, enter the following command to sync the database
if you start from scratch and dont have created a django application yet, you should do :
python manage.py syncdb
otherwise do :
python manage.py migrate
if you meet some errors with this last command, have a look at MIGRATION_0.10.x_to_0.11.x.rst file
python manage.py runserver
Now that everything is in place, Celery will do our job in background in the meantime you will be able to manage your triggers from the front part
Once the module is installed, go to the admin panel and activate the service you want.
Currently there are 5 services, Evernote, Pocket, RSS, Trello and Twitter.
Once they are activated....
... User can use them
The user activates the service for their own need. If the service requires an external authentication, he will be redirected to the service which will ask him the authorization to acces the user's account. Once it's done, goes back to django-trigger-happy to finish and record the "auth token".
a set of 3 pages will ask to the user information that will permit to trigger data from a service "provider" to a service "consummer".
For example :
- page 1 : the user gives a RSS feed
- page 2 : the user gives the name of the notebook where notes will be stored and a tag if he wants
- page 3 : the user gives a description
Here are the available management commands you can use by hand when you need to bypass the beat of Celery :
Available subcommands:
[django_th]
fire_read_data # will put date in cache
fire_publish_data # will read cache and publish data
To start handling the queue of triggers you/your users configured, just set those 2 management commands in a crontab or any other scheduler solution of your choice, if you dont want to use the beat of Celery
Also : Keep in mind to avoid to set a too short duration between 2 run to avoid to be blocked by the externals services (by their rate limitation) you/your users want to reach.
https://trigger-happy.readthedocs.org/
You can find all details of all existing services of the blog : https://www.foxmask.info/tag/TriggerHappy