Noisy result.

[JustinhoCHN]

The model was trained for 134 epochs, unpaired learning -- fivek and hdr dataset, I check the test_img for every epoch, almost like these noisy image:

All the hyperparams remain the same as the origin, can anyone give any insight about this situation? @nothinglo

[JeremyAlain]

Hi justin,
Maybe have a look at this bug i found, it might help!
#36

[nothinglo]

Please check your image format. It should match to the image reading method.

[JustinhoCHN]

ahhhh, thank you for your remind, I printed the image array (tiff format image), and get 20000~30000 pixel value, I think that's why I got the noisy result! @JeremyAlain

[JustinhoCHN]

So if I use cv2.imread(filename, -1), I'll get the uint16 data, that's tiff original pixel value, if I use cv2.imread(filename, 1) instead, the pixel value range from [0, 255], the precision will be reduced, will it affect the final result? @nothinglo

[nothinglo]

I think the precision is ok but you should make sure that after pre-processing you use [0, 1] range in the training.

[JustinhoCHN]

You already did the normalization step for us right? In DATA.py line 193 and 194:

self.input1_src = tf.placeholder(tf.as_dtype(FLAGS['data_input_dtype']), shape=[b, FLAGS['data_image_size'], FLAGS['data_image_size'], FLAGS['data_image_channel']])
self.input1 = tf.cast(self.input1_src, FLAGS['data_compute_dtype']) / self.input1_src.dtype.max

Especially in line 194, first casting the datatype from uint16 (tiff image) to np.float32 ('data_compute_dtype'), and then divided by 65535 ('self.input1_src.dtype.max'), it should be range from [0, 1]. We don't need to do any additional normalization step right? @nothinglo

Even reading the tiff image with cv2.imread(filename, 1), I still got these noisy test image, I'm sure the training set tiff images are readable. @JeremyAlain

[JustinhoCHN]

May I ask your tif images are 8 bits or 16 bits? @nothinglo

[JustinhoCHN]

I found the problem, if we export the fivek dataset to tif format in 16 bits using lightroom, opencv will write the images in 8 bits during the testing, the testing image will be noisy, we should prepare the 8 bits dataset rather then 16 bits. @JeremyAlain

Therefore the correct dataset preparation pipeline is:

1. import all fivek dataset into adobe lightroom.
2. export all images in the following settings:
   
3. if you want to use flickr hdr data, download them first, and then do the step 1 and 2.
4. if you want to use expert C's images as label, double click the /fivek_dataset/raw_photos/fivek.lrcat, it will load the whole modify record from 5 experts, and you should see the expert C's images in the left side, and then do the step 1 and 2 again:
   

Feel free to add it to readme, maybe it can help other person. @nothinglo

Here're my hdr result, this model works so great!

[JeremyAlain]

@JustinhoCHN Thank you very much for your elaborate explanation. i think @nothinglo should add this to the readme, because this is something I have also experienced a lot with and still have not found out which solution they use.

One thing I would like to ask,

1. @JustinhoCHN why do you exactly use ProPhotoRGB and not sRGB? I am not totally sure which one to use but I thought the more standard one is sRGB. Also if you want to transform the RGB color space into other color spaces such as CIELAB, you need it in sRGB.

2. @JustinhoCHN Can you confirm that for your experiments you also used InputAsShotZeroed for your original images? (as I describe here What are the original images?  #34)

[JustinhoCHN]

@JeremyAlain My understanding:

1. According to the official fivek dataset website, they export the images in ProPhotoRGB mode, so I do the same, but I haven't think of other mode yet, you can try and see what you'll got.
2. I'm not 100% sure, but the InputAsShotZeroed collection looks like the original images, so I use them as input.

[JeremyAlain]

1. You are right, I didn't see that.
2. OK good to know that you also use that.

[nothinglo]

Thanks all. I will add this issue to the README.

[xiaozhi2015]

Do you know how to expand the inference dimension to 2048?
@JustinhoCHN and @JeremyAlain
I'll appreciate it very much for your answers.

[TriLoo]

How to convert the result to sRGB ?

I used the ProPhoto RGB during training, and now I want to store my result during inference. How can I convert the results to sRGB color space ? thanks
@JustinhoCHN

[lvn27]

1. @JustinhoCHN why do you exactly use ProPhotoRGB and not sRGB? I am not totally sure which one to use but I thought the more standard one is sRGB. Also if you want to transform the RGB color space into other color spaces such as CIELAB, you need it in sRGB.

as for colourspaces :

• AdobeRGB is the standard for photographers and 'pro' digital printing
• sRGB is only used for the web and screens and recreational printing
• ProPhoto RGB is the biggest colourspace but unfortunately our eyes can't see the whole range, normal monitors can't show it and printers can't print it either. 16-bit is the standard for ProPhoto since it contains the most data.