This is a console programme which using Gumbo
, indicators
, cpr
, argparse
, termcolor
and cUrl
for scrape all characters card images, wish images and constellation images from Genshin Impact Wiki Fandom, as Tree Directory below, that all images file is contained into a CDN server. For the images it's always retrive the latest version and uncroped version. Docker image build are available.
Genshin Wiki Fandom (https://genshin-impact.fandom.com/wiki/Genshin_Impact_Wiki) └── ... ├── Wiki Fandom Character Category Page (https://genshin-impact.fandom.com/wiki/Category:Character_Cards) │ └── Character Page (https://genshin-impact.fandom.com/wiki/*Character_Name*) │ ├── ... │ ├── Character Card Image (Static_CDN_File [https://static.wikia.nocookie.net/gensin-impact/images/*unique_character*/*unique_number_character*/*Character_Name*_Card.png/revision/latest/]) │ ├── Character Wish Image (Static_CDN_File [https://static.wikia.nocookie.net/gensin-impact/images/*unique_character*/*unique_number_character*/*Character_Name*_Wish.png/revision/latest/]) │ └── ... ├── Wiki Fandom Constellations Category Page (https://genshin-impact.fandom.com/wiki/Category:Constellation_Overviews) │ └── Constellation Page (https://genshin-impact.fandom.com/wiki/*Constellation_Name*) │ ├── ... │ ├── Constellation Image (Static_CDN_File [https://static.wikia.nocookie.net/gensin-impact/images/*unique_character*/*unique_number_character*/*Constellation_Name*.png/revision/latest]) │ └── ... ├── Wiki Fandom Character Introduction Page (https://genshin-impact.fandom.com/wiki/Category:Character_Introduction_Cards) │ └── Character Media Page (https://genshin-impact.fandom.com/wiki/*character_name*/Media) │ ├── ... │ ├── Character Introduction Card Image (Static_CDN_File [https://static.wikia.nocookie.net/gensin-impact/images/*unique_character*/*unique_number_character*/*Character_Name*_Introduction.png/revision/latest/]) │ ├── Character Namecard Image (Static_CDN_File [https://static.wikia.nocookie.net/gensin-impact/images/*unique_character*/*unique_number_character*/*Name_Card_Name*.png/revision/latest/]) │ └── ... ├── Wiki Fandom Version page (https://genshin-impact.fandom.com/wiki/Version) │ ├── ... │ ├── Version Splashscreen Image (Static_CDN_File [https://static.wikia.nocookie.net/gensin-impact/images/*unique_character*/*unique_character_number*/Splashscreen_*Version_Name*.png/revision/latest/]) │ └── ... ├── Wiki Fandom TGC Character Card Page (https://genshin-impact.fandom.com/wiki/Category:Genius_Invokation_TCG_Character_Cards) │ ├── ... │ ├── Character TGC Card Image (Static_CDN_File [https://static.wikia.nocookie.net/gensin-impact/images/*unique_number*/*unique_character_number*/*Character_Name*_Character_Card.png/revision/latest/]) │ ├── Character Dymanics TGC Card Image (Static_CDN_File [https://static.wikia.nocookie.net/gensin-impact/images/*unique_character*/*unique_number_charac_character*/*character_name*.gif/revision/latest]) │ └── ... ├── Wiki Fandom Namecard Page (https://genshin-impact.fandom.com/wiki/Category:Character_Namecards) │ ├── ... │ ├── Character Namecard Background Image (Static_CDN_File [https://static.wikia.nocookie.net/gensin-impact/images/*unique_character*/*unique_number_charac_character*/*namecard_name*.png/revision/latest/]) │ ├── Battle Pass Namecard Background Image (Static_CDN_File [https://static.wikia.nocookie.net/gensin-impact/images/*unique_character*/*unique_number_charac_character*/*namecard_name*.png/revision/latest]) │ └── ... ├── Wiki Fandom Character Page (https://genshin-impact.fandom.com/wiki/wiki/Category:Playable_Characters) │ └── Character Page Gallery (https://genshin-impact.fandom.com/wiki/*character_name*/Gallery) │ ├── ... │ ├── Sticker Image (Static_CDN_File [https://static.wikia.nocookie.net/gensin-impact/images/*unique_number*/*unique_number_charac_character*/*sticker_name_type*.png/revision/latest/]) │ └── ... └── ...
- Scrap all characters link from
/wiki/Category:Character_Card
. - Scrap all characters card images link from each characters links to file.
- Get characters images card HD and uncropped.
- Download all characters card images.1
- Get characters images wish HD and uncropped.
- Download all characters wish images.2
- Scrap all characters link from
/wiki/Category:Constellation_Overviews
. - Scrap all constellation images link from each characters constellation link to file.
- Get characters constellation images HD and uncropped.3
- Scrap all characters introduction images link from
/wiki/Category:Character_Introduction_Cards
. - Scrap all characters introduction images link for each characters.
- Download all characters introduction images HD and uncropped.4
- Scrap all characters namecard images link from
/wiki/Category:Character_Namecards
. - Scrap all characters namecard images link for each characters.
- Download all characters namecard images HD and uncropped.5
- Adding internet check. 6
- Create seperated folder for characters Wish and Card images.
- Deploy into docker.7
- Compile for Linux (x86 and x64).
- Compile for ARM (armv7l and arm64).
- Compile for Android (armv7l, arm64, x86 and x64).
- Display file name when downloading file.
- Scrap all genshin version image link from
/wiki/Version
. - Download all version image HD and uncroppped.8
- Scrap all characters TCG card images link from
/wiki/Category:Character_Introduction_Cards
. - Scrap all characters TCG card images link for each characters.
- Download all characters TCG card images HD and uncropped.9
- Scrap all dynamics characters TCG card images link for each characters.
- Download all dynamics characters TCG card images HD and uncropped.10
- Scarp all sticker images link for each characters.
- Download all sticker images HD and uncropped.[^img_download_sticker]
-
Scarp all vision images link. -
Download all vision images HD and uncropped.11 - Scrap all battle pass namecard images link for each version.
- Download all battle pass namecard images HD and uncropped.12
- Scrap all birthday art images link for each version.
- Download all birthday art images HD and uncropped.13
- Make program can took command line arguments.
- Compile for macOS (x64 and arm64).
- Adding feature for download certain characters.
For who like to build this apps, you needed to download vcpkg
to get neccesary dependency, which can be done by visiting vcpkg website. After vcpkg
has been installed, the next step, you needed install all dependencies by using command below (assume your working directory is inside ../../vcpkg
):
vcpkg install gumbo cpr curl indicators argparse termcolor --triplet x64-{YOUR_OS}-static
or
vcpkg install gumbo cpr curl indicators argparse termcolor --triplet x86-{YOUR_OS}
Important For x86, please recheck if your compiler do support cross compiling. For vcpkg, installation command may be different in this guide please recheck with
./vcpkg help triplet
.
-
Linux build
After installing all dependencies, make a folder on the root of this project by usingmkdir
command:mkdir build && cd WebScrapperGenshinCharacterCard
. After that run command below:cmake -B ../build -S . -DCMAKE_TOOLCHAIN_FILE=[YOUR_VCPKG_PATH]/scripts/buildsystems/vcpkg.cmake
or (for 32-bit version, please be cautious when building x86)
cmake -B ../build -DBUILD_32=1 -S . -DCMAKE_TOOLCHAIN_FILE=[YOUR_VCPKG_PATH]/scripts/buildsystems/vcpkg.cmake
after
cmake
successfully generatemakefile
, go to build folder using this commandcd ../build && make all
or you can usecmake --build ../build
. After all progress has been completed you can find the executable in your working directory, for using the program you just simply run./wsgcc_{YOUR_OS}_{OS_ARCH}
e.g.: if build on architecturex86_64
and for Linux it will generate./wsgcc_linux_x64
. -
Windows Build
After installing all dependencies you need to run this commandvcpkg integrate install
in order Visual Studio can detect all dependencies. It's more recommended to compile using Visual Studio 2022. You can build usingx64
architecture, unless you already had downloadedx86
dependencies usingvcpkg
. The executable is located onD:\...\...\WebScrapperGenshinCharacterCard\wsgcc_win_x64.exe
, assuming your working directory inD:\
. -
Docker Build7
There are 2 methods for using Docker build:- Using dockerfile
if you want to build your own image you can use available docker file just simply run:After the image done building just simply run the image using:docker build --no-cache -t totmalone/wsgcc:latest -f ./Dockerfile .
If docker image successfully run, just simply rundocker run -it --rm -w /root/WSGCC/WebScrapperGenshinCharacterCard --name=wsgcc totmalone/wsgcc:latest ash
./wsgcc_{YOUR_OS}_{OS_ARCH}
if build on architecturex86_64
and for Linux it will generate./wsgcc_linux_x64
. To get desire image folder to your own device just run,mkdir genshin_images && docker cp wsgcc:/<folder_name> genshin_images/
. - Using Docker hub
If you want use pre-build image you just simply rundocker pull totmalone/wsgcc
, after docker image successfully pulled, just simply run the image using:If docker image successfully run, just simply rundocker run -it --rm -w /root/WSGCC/WebScrapperGenshinCharacterCard --name=wsgcc totmalone/wsgcc:latest ash
./wsgcc_{YOUR_OS}_{OS_ARCH}
if build on architecturex86_64
and for Linux it will generate./wsgcc_linux_x64
. To get desire image folder to your own device just run just run,mkdir genshin_images && docker cp wsgcc:/<folder_name> genshin_images/
.
- Using dockerfile
-
Android Build14
For building for android you need, installedvcpkg
dependencies which must be suited with your android device architecture, after all dependecies had been installed, you need to downloadandroid ndk
which you can get in ndk_downloads or you can usecurl -O https://dl.google.com/android/repository/android-ndk-{NDK_VERSION}-{YOUR OPERATING_SYSTEM}.zip
. After all dependecies had been downloaded you need toexport
all path for easier compiling and linking, for example:export VCPKG_ROOT=~/Documents/vcpkg && export export ANDROID_NDK_HOME=~/Documents/android-ndk-r25c && export vcpkg_toolchain_file=$VCPKG_ROOT/scripts/buildsystems/vcpkg.cmake && export android_toolchain_file=$ANDROID_NDK_HOME/build/cmake/android.toolchain.cmake && export android_abi=x86 && export vcpkg_target_triplet=x86-android
Note For
android_abi
andvcpkg_target_triplet
can be modified as needed, for reference microsoft guide for android abi.After that run command below:
cmake -B ../build -S . -DCMAKE_TOOLCHAIN_FILE=$vcpkg_toolchain_file -DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=$android_toolchain_file -DVCPKG_TARGET_TRIPLET=$vcpkg_target_triplet -DANDROID_ABI=$android_abi -DANDROID_PLATFORM=25 -DVCPKG_TARGET_ANDROID=ON
after
cmake
successfully generatemakefile
, go to build folder using this commandcd ../build && make all
or you can usecmake --build ../build
. After all completed the executable is in your working directory, for using the program just simply run./wsgcc_{YOUR_OS}_{OS_ARCH}
if build on android ABIarm64-v8a
it will generate./wsgcc_android_arm64
. You can run the compiled binary using termux, just simply clone this repository and run the binary file.Note For
termux
it isn't possible to build from there, because termux isn't supported advance Linux filesystem which may required to build this project. What tested isvcpkg
successful installed, butvcpkg
package failed to install duemake
unable to write file intotermux
.
For whom interested how this programme work, this project have a decent flowchart to follow with:
ASCII Tree Directory powered by: ASCII Tree Generator.
This project powered by: vcpkg, gumbo, cpr, cUrl, indicators, argparse, termcolor and C++20.
Data sources: genshin wiki fandom.
Footnotes
-
For characters card images are downloaded into a folder named
Character Genshin Card Image
. ↩ -
For characters wish images are downloaded into a folder named
Character Genshin Wish Image
. ↩ -
For characters constellation images are downloaded into a folder named
Character Genshin Constellation Image
. ↩ -
For characters introduction images are downloaded into a folder named
Character Genshin Introduction Card Image
. ↩ -
For characters namecard images are downloaded into a folder named
Character Genshin Namecard Image
. ↩ -
For macOS and Android implementation for internet check are limited, since this project are pure C++ and I don't want to call
system()
(it's already memory heavy).The function always returnThe function are usingtrue
.cUrl
to get response fromwww.google.com
. ↩ -
For genshin version images are downloaded into a folder named
Genshin Version Images
. ↩ -
For TCG characters card images are downloaded into a folder named
Genshin TGC Character Card Image
. ↩ -
For TCG dynamics characters card images are downloaded into a folder named
Genshin TGC Dynamics Character Card Image
. ↩ -
For genshin vision images are downloaded into a folder named. Due genshin wiki removed images of vision in theirs wiki, it cannot be downloaded anymore. ↩Genshin Vision Images
-
For battle pass namecard images are downloaded into a folder named
Genshin BP Namecard Image
. ↩ -
For birthday art are downloaded into a folder named
Genshin Charater Birthday Art
. ↩ -
For installing could use
termux
and. For now binary still broken and cannot be build after NDK update. ↩adb