A PyTorch implementation of the paper: [Adaptive Image Sampling using Deep Learning and its Application on X-Ray Fluorescence Image Reconstruction] [Arxiv Preprint] [IEEE Transactions on Multimedia]

Pipeline for adaptive sampling mask generation utilizing CNNs.

Visual Comparison of the reconstructed images using random, AIrS, KbAS, and NetM sampling masks at sampling rate c = 0.2. The first column is the input test image and the second column is the sampling mask, either random, AIrS, KbAS, or NetM, the third column is the sampled image obtained by the sampling mask, and the rest of the columns are the reconstruction results of NetE Inpainting, Harmonic Inpainting, Mumford-Shah Inpainting, and BPFA inpainting respectively.

Visualization of sampling and inpainting result of the “Bloemen en insecten” painting. (a) original RGB image with red bounding box. (b) region inside the bounding box of (a) for visualization purposes. (c), (g), (k), (o) and (s) random, AIrS, KbAS, Mascar and NetM sampling masks respectively. (d), (h), (l), (p) and (t) reconstruction results of each sampling mask using Harmonic algorithms. (e), (i), (m), (q) and (u) reconstruction results of each sampling mask using Mumford-Shah algorithm. (f), (j), (n), (r) and (v) reconstruction results of each sampling mask using BPFA algorithm. Computation time of each sampling mask and PSNR of the entirety of each reconstructed images are also shown.

• Anaconda
• PyTorch
• tqdm

pip install tqdm

• opencv

conda install -c conda-forge opencv

• tensorboard_logger

pip install tensorboard_logger

• h5py

conda install h5py

The train and val datasets are sampled from ImageNet. Train dataset has 100000 images. Val dataset has 1000 images. Download the datasets from here, and then extract it into $data directory. Modify the path of $data directory in line#48 of file train_NetE.py and line#48 of file train_NetM.py.

The test image dataset are sampled from ImageNet. It contains 100 images. It is stored in file data_val_100.h5 .

Run

python train_NetE.py

to train the image inpainting network NetE.

After NetE is trained, modify the file name of trained NetE in line#29 of file train_NetM.py and run

python train_NetM.py

to train the adaptive image sampling network NetM.

To visualize the training process, run

tensorboard --logdir tensorBoardRuns

python test_NetE_h5.py

The output reconstructed images are in results/netE_results directory.

python test_NetM_h5.py

The output reconstructed images are in results/netM_results directory.