Releases: OOXXXXOO/DARTH
version update 0.1.8
Feature update
Feature update
-
- remove the package version requirement
-
- compile support for arm64 apple silicon
-
- support for choosing channel on raster object
-
- add document with sphnix style for next version
GDAL Reference update
The version of gdal has change the reference of core function method like , from import gdal
to
from osgeo import gdal
from osgeo import ogr
from osgeo import osr
from osgeo import gdal_array
from osgeo import gdalconst
the new version has upload to pypi . Obviously, the China mainland user may can't achieve the google map server.
The recommend way is 'BIng Virture Earth' like:
>>> from darth.downloader import downloader
>>> Bing=downloader("Bing VirtualEarth",thread_count=8)
# ---------------------------------------------------------------------------- #
# ----------------- ██████╗ █████╗ ██████╗ ████████╗██╗ ██╗ ---------------- #
# ----------------- ██╔══██╗██╔══██╗██╔══██╗╚══██╔══╝██║ ██║ ---------------- #
# ----------------- ██║ ██║███████║██████╔╝ ██║ ███████║ ---------------- #
# ----------------- ██║ ██║██╔══██║██╔══██╗ ██║ ██╔══██║ ---------------- #
# ----------------- ██████╔╝██║ ██║██║ ██║ ██║ ██║ ██║ ---------------- #
# ----------------- ╚═════╝ ╚═╝ ╚═╝╚═╝ ╚═╝ ╚═╝ ╚═╝ ╚═╝ ---------------- #
# ---------------------------------------------------------------------------- #
# ---------------------------------------------------------------------------- #
# MAP Production Toolkit #
# ---------------------------------------------------------------------------- #
# ---------------------- MAP Serverv Init Successful by ---------------------- #
# ---------------------- Bing VirtualEarth ----------------------------------- #
>>> bing.add_cord(116.3, 39.9, 116.6, 39.7, 13)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'bing' is not defined
>>> Bing.add_cord(116.3, 39.9, 116.6, 39.7, 13)
# -----WGS BoundingBox: (116.27929689118595, 39.909736239191844, 116.63085939123488, 39.673370396403314)
# -----Mercator BoudingBox: (12944152.11792486, 4852834.051769256, 12983287.876406869, 4818590.263097501)
# -----Total tiles number:8 X 7
# -----Url Queue size: 56
>>> Bing.download()
95%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌ | 53/56 [00:02<00:00, 16.80it/s]# ===== Decode Downloading...
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 56/56 [00:00<00:00, 513962.85it/s]
# ------------------------------- Download Done ------------------------------ # | 0/56 [00:00<?, ?it/s]
# ===== Save description done Bing VirtualEarth-Mon Apr 18 05:16:52 2022-(116.3, 39.9, 116.6, 39.7)-13.json
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 56/56 [00:02<00:00, 22.13it/s]
>>>
Cli UI Update & Remove OPENCV requirement.
██████╗ █████╗ ██████╗ ████████╗██╗ ██╗
██╔══██╗██╔══██╗██╔══██╗╚══██╔══╝██║ ██║
██║ ██║███████║██████╔╝ ██║ ███████║
██║ ██║██╔══██║██╔══██╗ ██║ ██╔══██║
██████╔╝██║ ██║██║ ██║ ██║ ██║ ██║
╚═════╝ ╚═╝ ╚═╝╚═╝ ╚═╝ ╚═╝ ╚═╝ ╚═╝
Important update:
-
The OpenCV is a package with a lot of requirements with C++/C/ & share object that need hard configure process in some system environments.
So, I replace the OpenCV lib by Pillow that mean the installation process will faster and easier. -
The CLI UI system update 👍 you could see information clearly
HTTP request header bug fix
The Tile download request URL header in version 0.0.8 will cause the Google mt1 -mt6 & Google China resources has bad link error.
So, I release update version 0.0.9 that use new version header to boost process download.
Next version will update function new CLI UI system & use vector unit as download area limitation.
Documet
Require fix
The GDAL version limit is not friendly for user ,so i cancel the GDAL version require
Then i test on the new env the process code work perfect:
# ---------------------------------------------------------------------------- #
# DARTH #
# ---------------------------------------------------------------------------- #
# ===== Bucket para preview {'ak': 'XXXXX', 'sk': 'XXXXXX', 'server': 'http:https://obs.cn-north-4.myhuaweicloud.com', 'bn': 'XXXXX'}
# ---------------------------------------------------------------------------- #
# ---------------------------------- Step I ---------------------------------- #
# ---------------------------------------------------------------------------- #
# ---------------------------------------------------------------------------- #
# MAP Production Toolkit #
# ---------------------------------------------------------------------------- #
# ---------------------- MAP Serverv Init Successful by ---------------------- #
# ---------------------- Google China ---------------------------------------- #
# ---------------------------------------------------------------------------- #
# Bucket ToolKit #
# ---------------------------------------------------------------------------- #
# ----access key (AK) : ISSSVUVTXQWXSCPKR23N
# ----secret key (SK): IYZbHMxJss3vXsoi9pqArIySf205lPcoISmm6ReJ
# ----server : http:https://obs.cn-north-4.myhuaweicloud.com
# ----bucket name : obs-tq-dataset
# ----root : /
# ---------------------------------------------------------------------------- #
# ---------------------------------------------------------------------------- #
# Vector Toolkit #
# ---------------------------------------------------------------------------- #
# ---------------------------------------------------------------------------- #
# TIFF process Toolkit #
# ---------------------------------------------------------------------------- #
# -----Class TIF init without filename
# -----Valid vector format : mbtiles
# ----------------------------- Meta Information ----------------------------- #
# -----ZOOM_LEVEL : 14 #
# -----attribution : <a href="http:https://www.openmaptiles.org/" target="_blank">&co#
# -----center : 116.4025,39.92,14 #
# -----description : Extract from https://openmaptiles.org #
# -----maxzoom : 14 #
# -----minzoom : 0 #
# -----name : OpenMapTiles #
# -----pixel_scale : 256 #
# -----mtime : 1499626373833 #
# -----format : pbf #
# -----id : openmaptiles #
# -----version : 3.6.1 #
# -----maskLevel : 5 #
# -----bounds : 115.686,39.414,117.119,40.426 #
# -----planettime : 1499040000000 #
# -----basename : china_beijing.mbtiles #
# ----------------------------- Meta Information ----------------------------- #
# -----Description : /workspace/osm-2017-07-03-v3.6.1-china_beijing.mbtiles
# -----LayerCount: 15
# -----Layer : 0 LayerName : water
# -----Layer : 1 LayerName : waterway
# -----Layer : 2 LayerName : landcover
# -----Layer : 3 LayerName : landuse
# -----Layer : 4 LayerName : mountain_peak
# -----Layer : 5 LayerName : park
# -----Layer : 6 LayerName : boundary
# -----Layer : 7 LayerName : aeroway
# -----Layer : 8 LayerName : transportation
# -----Layer : 9 LayerName : building
# -----Layer : 10 LayerName : water_name
# -----Layer : 11 LayerName : transportation_name
# -----Layer : 12 LayerName : place
# -----Layer : 13 LayerName : housenumber
# -----Layer : 14 LayerName : poi
# -----Extent: (12878106.611910647, 13037627.442217408, 4781148.214470335, 4928041.927030869)
# -----Alread Load: /workspace/osm-2017-07-03-v3.6.1-china_beijing.mbtiles
# -------------------------------- DEFINE DONE ------------------------------- #
# ---------------------------------------------------------------------------- #
# ---------------------------------- Step II --------------------------------- #
# ---------------------------------------------------------------------------- #
# -----Set Default Layer | building | : <osgeo.ogr.Layer; proxy of <Swig Object of type 'OGRLayerShadow *' at 0x7fed250f3930> >
# -----WGS BoundingBox: (116.4930128805835, 39.9085118736554, 116.60310895291508, 39.790620416540385)
# -----Mercator BoudingBox: (12967942.873035664, 4852656.366560855, 12980198.711744247, 4835562.295251646)
# -----Total tiles number:5 X 7
# -----Url Queue size: 35
# -----Set filter Rect: (12967942.873035664, 4852656.366560855, 12980198.711744247, 4835562.295251646)
# ---------------------------------------------------------------------------- #
# --------------------------------- Step III --------------------------------- #
# ---------------------------------------------------------------------------- #
# ===== imagery dir : DataSet/images/
# ===== targets dir : DataSet/targets/
Getting bucket metadata
storageClass: STANDARD
accessContorlAllowOrigin: None
accessContorlMaxAge: None
accessContorlExposeHeaders: None
accessContorlAllowMethods: None
accessContorlAllowHeaders: None
Deleting bucket CORS
status204
# ===== Bucket imagery root : DataSets/DataSet/images/
# ===== Bucket Targets root : DataSets/DataSet/targets/
# ===== Bucket Description root : DataSets/DataSet/
# ===== Base Folder DataSets
# ===== list (DataSets):
# ===== object count : 1000
|--- : DataSets/ etag["d41d8cd98f00b204e9800998ecf8427e"]
|--- : DataSets/Building_Beijing_14/.meta etag["f802f58e7d89c2cad4f9d42d54a805e7"]
|--- : DataSets/Building_Beijing_14/Google China-Wed Jun 10 17:44:24 2020-(116.3, 39.9, 116.6, 39.7)-14.json etag["f5d4d06dde3c263cc7c08dd203e9ea86"]
|--- : DataSets/Building_Beijing_14/images/Google China-13484-6208-14.tif etag["62724a9ee3e6a089e4a8974e0aa2b115"]
|--- : DataSets/Building_Beijing_14/images/Google China-13484-6209-14.tif etag["1712a0eee45ef4aa0460b55918cbbd6b"]
|--- : DataSets/Building_Beijing_14/images/Google China-13484-6210-14.tif etag["4c3023113719fa50192bd379de31d843"]
|--- : DataSets/Building_Beijing_14/images/Google China-13484-6211-14.tif etag["59cb243b1932909164dafdeeb87eb29d"]
|--- : DataSets/Building_Beijing_14/images/Google China-13484-6212-14.tif etag["0e239caf1796f5b71245c50de3d2b535"]
|--- : DataSets/Building_Beijing_14/images/Google China-13484-6213-14.tif etag["df9e7aeb65eb9c559e52f7b7565a0c96"]
|--- : DataSets/Building_Beijing_14/images/Google China-13484-6214-14.tif etag["c4620def4d0157802b77acf59e0a4061"]
# ===== Start Downloading.....
100%|█████████████████████████████████████████████████████████████████████████████████| 35/35 [00:00<00:00, 15.94it/s]# ===== Decode Downloading...
100%|█████████████████████████████████████████████████████████████████████████████| 35/35 [00:00<00:00, 139411.81it/s]
# ------------------------------- Download Done ------------------------------ # | 0/35 [00:00<?, ?it/s]
# ===== Save description done DataSet/Google China-Tue Jun 30 11:15:39 2020-(116.5, 39.9, 116.6, 39.8)-14.json
100%|█████████████████████████████████████████████████████████████████████████████████| 35/35 [00:00<00:00, 38.12it/s]
# ===== Start Generate.....
100%|█████████████████████████████████████████████████████████████████████████████████| 35/35 [00:03<00:00, 11.54it/s]
# ---------------------------------------------------------------------------- #
# ---------------------------------- Step IV --------------------------------- #
# ---------------------------------------------------------------------------- #
# ===== Upload dataset meta DataSets/DataSet/.meta
# ===== Upload dataset description DataSets/DataSet/Google China-Tue Jun 30 11:15:39 2020-(116.5, 39.9, 116.6, 39.8)-14.json
# ===== upload imagry to bucket.....
100%|█████████████████████████████████████████████████████████████████████████████████| 35/35 [00:01<00:00, 25.76it/s]
# ===== upload target to bucket.....
100%|█████████████████████████████████████████████████████████████████████████████████| 35/35 [00:00<00:00, 36.51it/s]
# ===== uploaded bucket:
# ===== list (DataSets):
# ===== object count : 1000
|--- : DataSets/ etag["d41d8cd98f00b204e9800998ecf8427e"]
|--- : DataSets/Building_Beijing_14/.meta etag["f802f58e7d89c2cad4f9d42d54a805e7"]
|--- : DataSets/Building_Beijing_14/Google China-Wed Jun 10 17:44:24 2020-(116.3, 39.9, 116.6, 39.7)-14.json etag["f5d4d06dde3c263cc7c08dd203e9ea86"]
|--- : DataSets/Building_Beijing_14/images/Google China-13484-6208-14.tif etag["62724a9ee3e6a089e4a8974e0aa2b115"]
|--- : DataSets/Building_Beijing_14/images/Google China-13484-6209-14.tif etag["1712a0eee45ef4aa0460b55918cbbd6b"]
|--- : DataSets/Building_Beijing_14/images/Google China-13484-6210-14.tif etag["4c3023113719fa50192bd379de31d843"]
|--- : DataSets/Building_Beijing_14/images/Google China-13484-6211-14.tif etag["59cb243b1932909164dafdeeb87eb29d"]
|--- : DataSets/Building_Beijing_14/images/Google China-13484-6212-14.tif eta...
beta version python package
fix license problem & call problem the root call function has been turn back to low case 👍
from darth.process import Process
Test Target
Test Version
ak='XXXXX'
sk='XXXXX'
server='http:https://obs.cn-north-4.myhuaweicloud.com'
bucketname="XXXXX"
vector="/workspace/data/osm-2017-07-03-v3.6.1-china_beijing.mbtiles"
cord=(116.5, 39.9, 116.6, 39.8, 14)
class_key='building'
datasource="Google China"
DataSetName="BuildingDataSet"
from Darth.process import Process
Process(
vector,
cord,
class_key
# Upload=True,
# ak=ak,
# sk=sk,
# server=server,
# bn=bucketname
)
Test Sucessful & Package Could be download