Skip to content

Releases: OOXXXXOO/DARTH

version update 0.1.8

23 Feb 11:23
Compare
Choose a tag to compare

code format in 0.1.8 version

Feature update

29 Sep 17:47
70cc4f7
Compare
Choose a tag to compare

Feature update

    1. remove the package version requirement
    1. compile support for arm64 apple silicon
    1. support for choosing channel on raster object
    1. add document with sphnix style for next version

GDAL Reference update

17 Apr 21:42
Compare
Choose a tag to compare

The version of gdal has change the reference of core function method like , from import gdalto

from osgeo import gdal
from osgeo import ogr
from osgeo import osr
from osgeo import gdal_array
from osgeo import gdalconst

the new version has upload to pypi . Obviously, the China mainland user may can't achieve the google map server.
The recommend way is 'BIng Virture Earth' like:

>>> from darth.downloader import downloader
>>> Bing=downloader("Bing VirtualEarth",thread_count=8)

# ---------------------------------------------------------------------------- #
# ----------------- ██████╗  █████╗ ██████╗ ████████╗██╗  ██╗ ---------------- #
# ----------------- ██╔══██╗██╔══██╗██╔══██╗╚══██╔══╝██║  ██║ ---------------- #
# ----------------- ██║  ██║███████║██████╔╝   ██║   ███████║ ---------------- #
# ----------------- ██║  ██║██╔══██║██╔══██╗   ██║   ██╔══██║ ---------------- #
# ----------------- ██████╔╝██║  ██║██║  ██║   ██║   ██║  ██║ ---------------- #
# ----------------- ╚═════╝ ╚═╝  ╚═╝╚═╝  ╚═╝   ╚═╝   ╚═╝  ╚═╝ ---------------- #
# ---------------------------------------------------------------------------- #

# ---------------------------------------------------------------------------- #
#                            MAP Production Toolkit                            #
# ---------------------------------------------------------------------------- #
# ---------------------- MAP Serverv Init Successful by ---------------------- #
# ---------------------- Bing VirtualEarth ----------------------------------- #
>>> bing.add_cord(116.3, 39.9, 116.6, 39.7, 13)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'bing' is not defined
>>> Bing.add_cord(116.3, 39.9, 116.6, 39.7, 13)
# -----WGS BoundingBox: (116.27929689118595, 39.909736239191844, 116.63085939123488, 39.673370396403314)
# -----Mercator BoudingBox: (12944152.11792486, 4852834.051769256, 12983287.876406869, 4818590.263097501)
# -----Total tiles number:8 X 7
# -----Url Queue size: 56
>>> Bing.download()
 95%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌       | 53/56 [00:02<00:00, 16.80it/s]# ===== Decode Downloading...
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 56/56 [00:00<00:00, 513962.85it/s]
# ------------------------------- Download Done ------------------------------ #                                                                         | 0/56 [00:00<?, ?it/s]

# ===== Save description done Bing VirtualEarth-Mon Apr 18 05:16:52 2022-(116.3, 39.9, 116.6, 39.7)-13.json
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 56/56 [00:02<00:00, 22.13it/s]
>>>

Cli UI Update & Remove OPENCV requirement.

16 Sep 15:22
Compare
Choose a tag to compare
██████╗  █████╗ ██████╗ ████████╗██╗  ██╗
██╔══██╗██╔══██╗██╔══██╗╚══██╔══╝██║  ██║
██║  ██║███████║██████╔╝   ██║   ███████║
██║  ██║██╔══██║██╔══██╗   ██║   ██╔══██║
██████╔╝██║  ██║██║  ██║   ██║   ██║  ██║
╚═════╝ ╚═╝  ╚═╝╚═╝  ╚═╝   ╚═╝   ╚═╝  ╚═╝

Important update:

  1. The OpenCV is a package with a lot of requirements with C++/C/ & share object that need hard configure process in some system environments.
    So, I replace the OpenCV lib by Pillow that mean the installation process will faster and easier.

  2. The CLI UI system update 👍 you could see information clearly

HTTP request header bug fix

09 Sep 20:41
Compare
Choose a tag to compare

The Tile download request URL header in version 0.0.8 will cause the Google mt1 -mt6 & Google China resources has bad link error.
So, I release update version 0.0.9 that use new version header to boost process download.

Next version will update function new CLI UI system & use vector unit as download area limitation.

Documet

30 Jun 08:10
Compare
Choose a tag to compare

Full Document

Require fix

30 Jun 03:23
Compare
Choose a tag to compare

The GDAL version limit is not friendly for user ,so i cancel the GDAL version require

Then i test on the new env the process code work perfect:

# ---------------------------------------------------------------------------- #
#                                     DARTH                                    #
# ---------------------------------------------------------------------------- #
# ===== Bucket para preview {'ak': 'XXXXX', 'sk': 'XXXXXX', 'server': 'http:https://obs.cn-north-4.myhuaweicloud.com', 'bn': 'XXXXX'}



# ---------------------------------------------------------------------------- #
# ---------------------------------- Step I ---------------------------------- #
# ---------------------------------------------------------------------------- #
# ---------------------------------------------------------------------------- #
#                            MAP Production Toolkit                            #
# ---------------------------------------------------------------------------- #
# ---------------------- MAP Serverv Init Successful by ---------------------- #
# ---------------------- Google China ---------------------------------------- #
# ---------------------------------------------------------------------------- #
#                                Bucket ToolKit                                #
# ---------------------------------------------------------------------------- #
# ----access key (AK) :  ISSSVUVTXQWXSCPKR23N
# ----secret key (SK):  IYZbHMxJss3vXsoi9pqArIySf205lPcoISmm6ReJ
# ----server :  http:https://obs.cn-north-4.myhuaweicloud.com
# ----bucket name :  obs-tq-dataset
# ----root :  /
# ---------------------------------------------------------------------------- #
# ---------------------------------------------------------------------------- #
#                                Vector Toolkit                                #
# ---------------------------------------------------------------------------- #
# ---------------------------------------------------------------------------- #
#                            TIFF process Toolkit                              #
# ---------------------------------------------------------------------------- #
# -----Class TIF init without filename
# -----Valid vector format : mbtiles

# ----------------------------- Meta Information ----------------------------- #
# -----ZOOM_LEVEL : 14                                                         #
# -----attribution : <a href="http:https://www.openmaptiles.org/" target="_blank">&co#
# -----center : 116.4025,39.92,14                                              #
# -----description : Extract from https://openmaptiles.org                     #
# -----maxzoom : 14                                                            #
# -----minzoom : 0                                                             #
# -----name : OpenMapTiles                                                     #
# -----pixel_scale : 256                                                       #
# -----mtime : 1499626373833                                                   #
# -----format : pbf                                                            #
# -----id : openmaptiles                                                       #
# -----version : 3.6.1                                                         #
# -----maskLevel : 5                                                           #
# -----bounds : 115.686,39.414,117.119,40.426                                  #
# -----planettime : 1499040000000                                              #
# -----basename : china_beijing.mbtiles                                        #
# ----------------------------- Meta Information ----------------------------- #

# -----Description :  /workspace/osm-2017-07-03-v3.6.1-china_beijing.mbtiles
# -----LayerCount: 15
# -----Layer : 0  LayerName :  water
# -----Layer : 1  LayerName :  waterway
# -----Layer : 2  LayerName :  landcover
# -----Layer : 3  LayerName :  landuse
# -----Layer : 4  LayerName :  mountain_peak
# -----Layer : 5  LayerName :  park
# -----Layer : 6  LayerName :  boundary
# -----Layer : 7  LayerName :  aeroway
# -----Layer : 8  LayerName :  transportation
# -----Layer : 9  LayerName :  building
# -----Layer : 10  LayerName :  water_name
# -----Layer : 11  LayerName :  transportation_name
# -----Layer : 12  LayerName :  place
# -----Layer : 13  LayerName :  housenumber
# -----Layer : 14  LayerName :  poi
# -----Extent: (12878106.611910647, 13037627.442217408, 4781148.214470335, 4928041.927030869)
# -----Alread Load: /workspace/osm-2017-07-03-v3.6.1-china_beijing.mbtiles
# -------------------------------- DEFINE DONE ------------------------------- #



# ---------------------------------------------------------------------------- #
# ---------------------------------- Step II --------------------------------- #
# ---------------------------------------------------------------------------- #
# -----Set Default Layer | building | :  <osgeo.ogr.Layer; proxy of <Swig Object of type 'OGRLayerShadow *' at 0x7fed250f3930> >
# -----WGS BoundingBox: (116.4930128805835, 39.9085118736554, 116.60310895291508, 39.790620416540385)
# -----Mercator BoudingBox: (12967942.873035664, 4852656.366560855, 12980198.711744247, 4835562.295251646)
# -----Total tiles number:5 X 7
# -----Url Queue size: 35
# -----Set filter Rect: (12967942.873035664, 4852656.366560855, 12980198.711744247, 4835562.295251646)



# ---------------------------------------------------------------------------- #
# --------------------------------- Step III --------------------------------- #
# ---------------------------------------------------------------------------- #
# ===== imagery dir : DataSet/images/
# ===== targets dir : DataSet/targets/
Getting bucket metadata

storageClass: STANDARD
accessContorlAllowOrigin: None
accessContorlMaxAge: None
accessContorlExposeHeaders: None
accessContorlAllowMethods: None
accessContorlAllowHeaders: None
Deleting bucket CORS

status204
# ===== Bucket imagery root  : DataSets/DataSet/images/
# ===== Bucket Targets root  : DataSets/DataSet/targets/
# ===== Bucket Description root : DataSets/DataSet/
# ===== Base Folder DataSets
# ===== list (DataSets):
# ===== object count :  1000
   |--- : DataSets/ etag["d41d8cd98f00b204e9800998ecf8427e"]
   |--- : DataSets/Building_Beijing_14/.meta etag["f802f58e7d89c2cad4f9d42d54a805e7"]
   |--- : DataSets/Building_Beijing_14/Google China-Wed Jun 10 17:44:24 2020-(116.3, 39.9, 116.6, 39.7)-14.json etag["f5d4d06dde3c263cc7c08dd203e9ea86"]
   |--- : DataSets/Building_Beijing_14/images/Google China-13484-6208-14.tif etag["62724a9ee3e6a089e4a8974e0aa2b115"]
   |--- : DataSets/Building_Beijing_14/images/Google China-13484-6209-14.tif etag["1712a0eee45ef4aa0460b55918cbbd6b"]
   |--- : DataSets/Building_Beijing_14/images/Google China-13484-6210-14.tif etag["4c3023113719fa50192bd379de31d843"]
   |--- : DataSets/Building_Beijing_14/images/Google China-13484-6211-14.tif etag["59cb243b1932909164dafdeeb87eb29d"]
   |--- : DataSets/Building_Beijing_14/images/Google China-13484-6212-14.tif etag["0e239caf1796f5b71245c50de3d2b535"]
   |--- : DataSets/Building_Beijing_14/images/Google China-13484-6213-14.tif etag["df9e7aeb65eb9c559e52f7b7565a0c96"]
   |--- : DataSets/Building_Beijing_14/images/Google China-13484-6214-14.tif etag["c4620def4d0157802b77acf59e0a4061"]
# ===== Start Downloading.....
100%|█████████████████████████████████████████████████████████████████████████████████| 35/35 [00:00<00:00, 15.94it/s]# ===== Decode Downloading...
100%|█████████████████████████████████████████████████████████████████████████████| 35/35 [00:00<00:00, 139411.81it/s]
# ------------------------------- Download Done ------------------------------ #               | 0/35 [00:00<?, ?it/s]

# ===== Save description done DataSet/Google China-Tue Jun 30 11:15:39 2020-(116.5, 39.9, 116.6, 39.8)-14.json
100%|█████████████████████████████████████████████████████████████████████████████████| 35/35 [00:00<00:00, 38.12it/s]
# ===== Start Generate.....
100%|█████████████████████████████████████████████████████████████████████████████████| 35/35 [00:03<00:00, 11.54it/s]



# ---------------------------------------------------------------------------- #
# ---------------------------------- Step IV --------------------------------- #
# ---------------------------------------------------------------------------- #
# ===== Upload dataset meta DataSets/DataSet/.meta
# ===== Upload dataset description DataSets/DataSet/Google China-Tue Jun 30 11:15:39 2020-(116.5, 39.9, 116.6, 39.8)-14.json
# ===== upload imagry to bucket.....
100%|█████████████████████████████████████████████████████████████████████████████████| 35/35 [00:01<00:00, 25.76it/s]
# ===== upload target to bucket.....
100%|█████████████████████████████████████████████████████████████████████████████████| 35/35 [00:00<00:00, 36.51it/s]
# ===== uploaded bucket:
# ===== list (DataSets):
# ===== object count :  1000
   |--- : DataSets/ etag["d41d8cd98f00b204e9800998ecf8427e"]
   |--- : DataSets/Building_Beijing_14/.meta etag["f802f58e7d89c2cad4f9d42d54a805e7"]
   |--- : DataSets/Building_Beijing_14/Google China-Wed Jun 10 17:44:24 2020-(116.3, 39.9, 116.6, 39.7)-14.json etag["f5d4d06dde3c263cc7c08dd203e9ea86"]
   |--- : DataSets/Building_Beijing_14/images/Google China-13484-6208-14.tif etag["62724a9ee3e6a089e4a8974e0aa2b115"]
   |--- : DataSets/Building_Beijing_14/images/Google China-13484-6209-14.tif etag["1712a0eee45ef4aa0460b55918cbbd6b"]
   |--- : DataSets/Building_Beijing_14/images/Google China-13484-6210-14.tif etag["4c3023113719fa50192bd379de31d843"]
   |--- : DataSets/Building_Beijing_14/images/Google China-13484-6211-14.tif etag["59cb243b1932909164dafdeeb87eb29d"]
   |--- : DataSets/Building_Beijing_14/images/Google China-13484-6212-14.tif eta...
Read more

beta version python package

29 Jun 20:25
Compare
Choose a tag to compare

fix license problem & call problem the root call function has been turn back to low case 👍

from darth.process import Process

Test Target

29 Jun 18:07
Compare
Choose a tag to compare
Test Target Pre-release
Pre-release

Test Version

ak='XXXXX'
sk='XXXXX'
server='http:https://obs.cn-north-4.myhuaweicloud.com'
bucketname="XXXXX"


vector="/workspace/data/osm-2017-07-03-v3.6.1-china_beijing.mbtiles"
cord=(116.5, 39.9, 116.6, 39.8, 14)
class_key='building'
datasource="Google China"
DataSetName="BuildingDataSet"

from Darth.process import Process

Process(
    vector,
    cord,
    class_key
    # Upload=True,
    # ak=ak,
    # sk=sk,
    # server=server,
    # bn=bucketname
)

Test Sucessful & Package Could be download