Currently, Lilac is A Distributed Scheduler Task System.
- Distributed scheduler, You can run many schedulers (Yes! it can on diffrent servers) at same time.
- Multi-threaded task execution (Leader/Follower variant)
- Scheduled event at a certain time or periodic execution like a crontab
- Attempting tasks that fail
- A thread-safe DB Api, and supports master/slave mode
- Take advantage of data-mapper pattern
- A Web Management Tool (Built in MVC pattern with solo and db )
- Add mongodb supports for scheduler task backend
- Improve web tool
- Add Loop Until event for task
- Add Queued task supports (Maybe zeromq or rabbitmq backend)
The modules are required to run lilac as below:
Firstly download or fetch it form github then run the command in shell:
cd lilac # the path to the project
python setup.py install
Built and tested under Python 2.7
Fork or download it, then run:
cd lilac # the path to the project
python setup.py develop
- create database in mysql:
- then run the mysql schema.sql script in the project directoy schema:
mysql -u yourusername -p yourpassword yourdatabase < schema.sql
if your database has not been created yet, log into your mysql first using:
mysql -u yourusername -p yourpassword yourdatabase
mysql>CREATE DATABASE a_new_database_name
# = you can =
mysql> USE a_new_database_name
mysql> source schema.sql
import logging
from lilac.server import LilacWebServer
import os.path
import db
db.setup({ 'host': 'localhost', 'user': 'test', 'passwd': 'test', 'db': 'lilac'})
def run(host='localhost', port=8080, debug=False):
setdebug(debug)
LilacWebServer(host=host,
port=port,
mako_cache_dir=os.path.join(os.path.dirname(os.path.dirname(__file__)), 'cache'),
debug=debug).serve_forever()
def setdebug(debug=False):
level = logging.DEBUG if debug else logging.INFO
logging.basicConfig(level=level,
format='%(asctime)s %(levelname)-8s %(message)s',
datefmt='%Y-%m-%d %H:%M:%S', filemode='a+')
if __name__ == '__main__':
run(debug=True)
when firstly run the web tool, please use the root account:
username: | lilac |
---|---|
password: | lilac |
then login the web tool and change your root password.
Note
if you wanna use gevent, please install gevent
firstly, then set use_gevent
to True
Here Is A Demo :
import db
from datetime import datetime
import urllib2
from lilac.app import App
from lilac.scheduler import Scheduler
import logging
LOGGER = logging.getLogger(__name__)
if __name__ == '__main__':
def get_date(url, session='xxx'):
date = None
try:
r = urllib2.urlopen(url)
date = r.info().dict['date']
except:
LOGGER.info('open failed')
LOGGER.info('session: %s, date:%s,', session, date)
def setdebug(debug=False):
level = logging.DEBUG if debug else logging.INFO
logging.basicConfig(level=level,
format='%(asctime)s %(levelname)-8s %(message)s',
datefmt='%Y-%m-%d %H:%M:%S', filemode='a+')
setdebug(False)
db.setup({ 'host': 'localhost', 'user': 'test', 'passwd': 'test', 'db': 'lilac'})
app = App()
app.add_task('task.test', get_date)
scheduler = Scheduler(app, 20, 20, 100)
db.execute('delete from cron')
for i in range(100):
if i % 2 == 0:
print i
action = 'task.not_found'
else:
action = 'task.test'
scheduler.add_task('name_%d' %(i), 'every 2', action, datetime.now(), 'https://www.google.com', session=i)
scheduler.run()
When you add job to scheduler, you see a event arugement. it is a specfic when to run task. Current event supports three types:
this event will only run once, in a future datetime, it should at least 1 minute speed from now: the pattern as below:
at %Y%m%d%H%M
this event will run in loop by minute(s), the pattern is a unsiged integer:
every minute(s)
this event pattern is pattern of crontab, current supports:
field allowed values ----- -------------- minute 0-59 hour 0-23 day of month 1-31 month 1-12 day of week 0-7
and the every sub pattern only support below regex expression format:
^(\d+-\d+/\d+)|(\d+-\d+)|(\d+)$
Copyright (C) 2014-2015 Thomas Huang
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, version 2 of the License.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program. If not, see <https://www.gnu.org/licenses/>.