• Nauval Muhammad Firdaus (A2005F0453) - GitHub@NauvalNC - Mobile Development - Universitas Bina Nusantara
• Fahrizza Irham Taufany (M2116L1465) - GitHub@fahrizzairham - Machine Learning - Institut Teknologi Kalimantan
• Muhammad Izzah Alfatih (M2012F1299) - GitHub@IzzahAlfatih - Machine Learning - Universitas Telkom
• Dimas Aji Permadi (C2152F1678) - GitHub@DimasAji1999 - Cloud Computing - Sekolah Tinggi Teknologi Bandung

Wacayang is a mobile application that able to identify Indonesian wayang kulit characters. This app uses images uploaded by the user as input, and the information about the name, description, image, and related video about the identified wayang character will be displayed on the app.

https://youtu.be/u1vkZIZzopQ

Link: https://drive.google.com/file/d/17EXgCPaWJuxIVNsX-qUjfb0lgfdHY-oy/view?usp=sharing
Alternative Link: https://drive.google.com/file/d/1hXyMuBRKxxPgjiQkxqbVkO4Q_6UlPJuB/view?usp=sharing

https://www.figma.com/file/JG7eT6Ri8gWlf9k0bcldGp/Wacayang

https://documenter.getpostman.com/view/20994859/UyxqDPV6

This app use wayang images provided by Wikimedia on Public Domain Licences, and videos from YouTube with attribution. For details, please refer to this link.

https://docs.google.com/spreadsheets/d/1wiKW1goLW4FmGMa_j7DgjjLb4kS6Mh9Tbfq9UIT5xsY/edit?usp=sharing

• Search various Indonesia wayang by typing on search bar.
• Upload image from camera or gallery to predict its wayang character using machine learning.
• Information about Indonesian wayangs inclunding name, story, images, and related wayang shows from local Indonesian puppeter.
• Personalization to add and remove wayang to/from favorite library.
• Share thoughts and knowledge through posting comments.
• Secured account sign in using Google Firebase JWT token.

• Android Studio IDE to develop the Android application.
• Google Cloud Platform as deployment platform for REST API, database, and ML model.
• Tensorflow and Keras to develop, train, and deploy ML model.

Wacayang consists of three components, Android, Cloud Computing, and Machine Learning. Basically, to integrate these, Cloud Computing acts as service to bridge communication between Android and Machine Learning. Here is a simple illustration on how our integration method works. Integration method explanation:

1. Android app send a network request using Retrofit library. This request has JWT token as Authorization header.
2. Cloud Run acts as service to serve request from the app. It will verify the token first before proceed.
3. After token is verified, Cloud Run services will access back-end app depending on the request. If it looking for user or wayang information, it will query to SQL database. If it looking for wayang image prediction, then it will post that image to ML model.
4. After back-end app finished processing, Cloud Run services will return the result as JSON literals to the Android app.
5. Android app will process the JSON literals and show relevant information to the user.

Wacayang Android app is developed using Android Studio IDE. Here are components that we used.

• Developed using Kotlin language.
• Composed by Activity and Fragment.
• Using RecycleView and its adapater for item listing.
• CameraX to utilize mobile camera features including flash, front/back camera, and more.
• Using Retrofit library for network request.
• Using ExoPlayer to play videos.
• BottomNavigation to navigate between main menus (Home, Favorite, and Settings).
• Connected to Firebase Auth for Google and anonymous sign in.
• Utilize LiveData, ViewModel, and Repository pattern for Single Source of Truth (SSOT)

• Android Studio Bumblebee 2021.1.1 Patch 2
• Minimum Android SDK 21

git clone https://github.com/Wacayang-Bangkit-2022/Wacayang-MobileDev.git

• Head to your Firebase Console.
• Then create or use your existing Firebase project.
• Activate Firebase Authentication feature.
• Open Project Settings -> General, select New App, and choose Android app.
• Fill debug SHA1 fingerprint. You can find this by execute Gradle -> signingReport on Android Studio.
• After the new Firebase app added, you will see your debug SHA1 added to that Firebase app. You can add another SHA1 such as your signed app SHA1. You can find your signed app SHA1 by executing this command on terminal. But, you will need to create your Keystore first

keytool -list -v -keystore <your keystore path> -alias <your alias>

• Install Firebase SDK to your Android Studio project.
• Download the google-service.json from your Firebase Console, and copy it to the app folder of your Android Studio project.

After you open the project, wait for the Gradle to finish building first. Then you can choose to build debug app by using Run -> Run'app'. Or you can build signed App by head to Build -> Generate Signed Bundle/APK.

• SQL database running on Google Cloud Platform
• REST API developed using Node.JS, Express, and Flask.
• Deployed REST API running as service on Google Cloud Run.

git clone https://github.com/Wacayang-Bangkit-2022/Wacayang_CloudComputing.git

• Open your last Firebase project used for Android Studio project installation
• Go to Project Settings -> Service Accounts
• Choose Generate New Private Key, and your private key JSON file will be downloaded.
• Rename your private key as serviceAccountKey.json
• Copy that file to your cloned project, both inside wacayang_general_api and wacayang_ai_api folder.

• Open your Cloud Console, head to SQL -> MySQL -> Create New Instance
• Create new database on your newly created SQL instance.
• Create necessary tables as showed on this schema below.
• Setup your database connection such as DB_USER, DB_PASS on the Connection tab.

• There will be two service running on Cloud Run.
• First, go to wacayang_general_api folder via terminal. And run gcloud run deploy command. Fill the rest of required fields. When it finish, it should shows your deployed service URL.
• Do the same for wacayang_ai_api folder, run gcloud run deploy command. Fill the rest of required fields. When it finish, it should shows your deployed service URL.
• These URL will be used by Android app to send network request. Replace the URL on ApiService.kt inside your Android Studio project to make it works with your deployed API.
• Go to your Cloud Console then head to your Cloud Run tab.
• Open your deployed wacayang_general_api service and create new revision by choosing Edit & Deploy New Revision.
• On Variable and Secrets tab, add new variable for DB_NAME, DB_USER, DB_PASS, and INSTANCE_CONNECTION_NAME. This necessary so your service can connect to your database. Fill these information based on your SQL database instance, then choose Deploy to create new revision.

Please head to this link for our detailed REST API documentation.

https://documenter.getpostman.com/view/20994859/UyxqDPV6

• Image Pre-processing
• Image Augmentation
• Early Stopping
• Callbacks
• Model Checkpoint
• Convolutional Neural Network (CNN)
• Transfer Learning
  • DenseNet121
  • InceptionV3
  • ResNet152V2
• Model Evaluate
• Accuracy and Loss Graph

• Google Colaboratory or Jupyter Notebook.
• Kaggle API Token.
• Latest Tensorflow Version 2.8.2.
• Python Version 3.6 or above.

• Wayang Image Dataset

Dataset Preview

(Left to right) Bagong, Cepot, Gareng, Petruk, and Semar

1. Generate Kaggle API token to get kaggle.json file
2. Open the .ipynb file in Google Colab or Jupyter Notebook:
   • Baseline Model (CNN)
   • DenseNet121
   • InceptionV3
   • ResNet152V2
3. Click Copy to Drive or Click File > Save a copy in Drive. This will allow you to run and edit the .ipynb file in your own Google Drive account
4. Upload your kaggle.json file (API Token)
5. Run every cell in the .ipynb file
6. Download the model .h5 file by left clicking the .h5file in the Colab directory (Automatically saved to /content/ file by Model Checkpoint)