# modify constants.py last 2 line, then run
python commic.py
# modify catalog.json, then run, then pick the id.
python commic.py -l
New version.
- Old version is not supported, sicne the page url was replaced by js. New version use selenium instead.
- Add "download from last stop page"
- Add "retry the failed page"
- Add "shutdown Windows system after completed"
- Modify "find_element"
- Find dictionary matching value in list
# sub = {'name':'a'}, vols = [sub1, sub2...]
tempinfo = next((sub for sub in vols if sub['name'] == lastinfo[1]), None)
- Jump to page by click the drop down list
if startpage > 1:
browser.find_element_by_xpath('//*[@id="page-selector"]/option['+str(startpage)+']').click()
- Read and overwrite file
# errorlog will be overwriten by tenmplog[]
with open(errorlog, 'r+', encoding='utf-8') as f:
templog = []
for x in f:
try:
whatever
except:
templog.append(x)
f.seek(0) # jump to line 1
f.writelines(templog) # insert new lines
f.truncate() # clean the rest lines
- Boolean arguments
# python x.py -s -r
...
def parse_args():
parser = argparse.ArgumentParser()
parser.add_argument('-s', action='store_true', help='shutdown system')
parser.add_argument('-r', action='store_true', help='down again from error log')
return parser.parse_args()
...
if args.r:
whatever
...
- Shutdown system, Windows
os.system("shutdown /s /f /t 1")
- How to cralwler when js is found.