0
$\begingroup$

I am trying to implement a patch creation function with using tensorflow's extract_image_patches function but dynamic output shape is not same as my expectation. Let me tell briefly what it does. Input shape is supposed to be 6000x4000. We first find its greatest common denominator. It turns out it is 3. then we pass '64' argument to our function to create patches with size of 3x64,2x64=192,128. This returns us 31x31 distinct patches. Everything works ok with static output, but when it comes to dynamic output things are not ok. I could not find which part caused a different dynamic output.

#       input_shape_inbuild:  (None, 6000, 4000, 1)
#         ---LAYER---
#         Input Size: (None, 6000, 4000, 1)
#         Patch Size: (x,y) = 192, 128
#         Aspect ratio: (3, 2)


!wget https://www.fujifilm.com/products/digital_cameras/x/fujifilm_x_pro2/sample_images/img/index/ff_x_pro2_001.JPG
img = cv2.imread('ff_x_pro2_001.JPG', 0)  
img = tf.reshape(img, [1,img.shape[0],img.shape[1],1])   



                
***tensorflow takes images as (y, x)
so a 6000x4000 im is given as tf.func(4000, 6000)


# Here I define custom layer in tensorflow.
class create_patches(Layer):
    
    def __init__(self, patchMultiplier):
        super(create_patches, self).__init__()
        self.patchMultiplier = patchMultiplier

    
    def build(self, input_shape):
        
        print('input_shape_inbuild: ', input_shape)
        def aspect_ratio(width, height):
            #find greatest common divider of input_shape
            def gcd(x, y):
                while y != 0:
                    (x, y) = (y, x % y)
                return x
            
            r = gcd(width, height)
            x = int(width/r)
            y = int(height/r)
            return x, y
        
        self.aspect_ratio = aspect_ratio(input_shape[1], input_shape[2])
        self.patchSize_x = self.aspect_ratio[0] * self.patchMultiplier
        self.patchSize_y = self.aspect_ratio[1] * self.patchMultiplier


    def call(self, inputs):
        
        print('---LAYER---')
        
        print('Input Size:', inputs._keras_shape)
        
        print('Patch Size: (x,y) = {}, {}'.format(self.patchSize_x, self.patchSize_y))
        
        print('Aspect ratio: {}'.format(self.aspect_ratio))
        
        print('---LAYER---')
    #call tf.extract_image_patches to return it.
        out = tf.extract_image_patches(images=inputs, 
                                        ksizes=[1, self.patchSize_y, self.patchSize_x, 1], 
                                        strides=[1, self.patchSize_y, self.patchSize_x, 1], 
                                        rates=[1, 1, 1, 1], 
                                        padding='VALID')

        
        return out
    
    def compute_output_shape(self, input_shape):

        """
        ksize_cols = patchSize_x
        ksize_rows = patchSize_y
        """

        #output shape=[batch, out_rows, out_cols, ksize_rows * ksize_cols * depth]

        """
        shape = (self.patchSize_x, self.patchSize_y, 
                (input_shape[1]/self.patchSize_x) * (input_shape[2]/self.patchSize_y))
        """

        shape =(input_shape[0],
                input_shape[1]/self.patchSize_x, # patch row count
                input_shape[2]/self.patchSize_y, # patch col count
                self.patchSize_x * self.patchSize_y) # patch pixel count


        return shape

    


#here is input with 6000x4000 pixels.
input_shape_1 = Input(shape=(6000, 4000, 1))
#here I fed input to my custom layer.
x1 = create_patches(64)(input_shape_1)
print('Output shape: ', x1.shape)

# here I build a model to see static output
f = K.function([input_shape_1], [x1])



import numpy as np
#result = f([np.random.randint(256, size=(1,4000,6000,1))])
result = f([img])


#print(result)
result = np.array(result)

# [batch, out_rows, out_cols, ksize_rows * ksize_cols * depth]
# Result shape:  (1, 1, 125, 125, 1536) 
print('Result shape: ', result.shape, '\n\n')

#print(result[:, :, :, 0].shape)

here is output I get.

input_shape_inbuild:  (None, 6000, 4000, 1)
---LAYER---
Input Size: (None, 6000, 4000, 1)
Patch Size: (x,y) = 192, 128
Aspect ratio: (3, 2)
---LAYER---
Output shape:  (?, 46, 20, 24576)
Result shape:  (1, 1, 31, 31, 24576) 
#####Result Shape is as I expected but at output shape I could not resolve where 46 and 20 come from. Could you tell me why it is like this?
$\endgroup$
2
  • $\begingroup$ I am unable to obtain 3 as the GCD of 6000 and 4000: their GCD is 2000 (and 3 doesn't even divide 4000). $\endgroup$
    – whuber
    Jul 30 '19 at 13:11
  • $\begingroup$ @whuber Ah sorry, I meant aspect ratio is 3/2, gcd is 2000 obviously. My mistake. $\endgroup$ Jul 31 '19 at 11:32
0
$\begingroup$

I think maybe you're just swapping the x and y axis.

With your selection of patch size and strides:

out = tf.extract_image_patches(images=inputs, 
                               ksizes=[1, self.patchSize_y, self.patchSize_x, 1], 
                               strides=[1, self.patchSize_y, self.patchSize_x, 1],
                               ...

I would expect the image dimensions to be:

6000//self.patchSize_y = 6000//128 = round(46.875) = 46
4000//self.patchSize_x = 4000//196 = round(20.408) = 20
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.