[Solved]: ModuleNotFoundError: No module named 'minecart'
You can solve this by installing minecart.
pip install minecart
Home Archives for January 2020
You can solve this by installing minecart.
pip install minecart
You can resolve this by installing PyMuPDF.
pip install PyMuPDF
Dropout is meant to block information from certain neurons completely to make sure the neurons do not co-adapt. So, the batch normalization has to be after dropout otherwise you are passing information through normalization statistics.
If you think about it, in typical ML problems, this is the reason we don’t compute mean and standard deviation over entire data and then split it into train, test and validation sets. We split and then compute the statistics over the train set and use them to normalize and centre the validation and test datasets
so I suggest Scheme 1
-> CONV/FC -> ReLu(or other activation) -> Dropout -> BatchNorm -> CONV/FC
as opposed to Scheme 2
-> CONV/FC -> BatchNorm -> ReLu(or other activation) -> Dropout -> CONV/FC
Please note that this means that the network under Scheme 2 should show over-fitting as compared to network under Scheme 1 but OP ran some tests as mentioned in question and they support Scheme 2
Let’s first look at how the number of learnable parameters is calculated for each individual type of layer you have, and then calculate the number of parameters in your example.
l
feature maps at the input, and has k
feature maps as output. The filter size is n
x m
. For example, this will look like this:Here, the input has l=32
feature maps as input, k=64
feature maps as output, and the filter size is n=3
x m=3
. It is important to understand, that we don’t simply have a 3×3 filter, but actually a 3x3x32 filter, as our input has 32 dimensions. And we learn 64 different 3x3x32 filters. Thus, the total number of weights is n*m*k*l
. Then, there is also a bias term for each feature map, so we have a total number of parameters of (n*m*l+1)*k
.
n
inputs and m
outputs, the number of weights is n*m
. Additionally, you have a bias for each output node, so you are at (n+1)*m
parameters.(n+1)*m
parameters, where n
is the number of inputs and m
is the number of outputs.The final difficulty is the first fully-connected layer: we do not know the dimensionality of the input to that layer, as it is a convolutional layer. To calculate it, we have to start with the size of the input image, and calculate the size of each convolutional layer. In your case, Lasagne already calculates this for you and reports the sizes – which makes it easy for us. If you have to calculate the size of each layer yourself, it’s a bit more complicated:
input_size - (filter_size - 1)
, in your case: 28 – 4 = 24. This is due to the nature of the convolution: we use e.g. a 5×5 neighborhood to calculate a point – but the two outermost rows and columns don’t have a 5×5 neighborhood, so we can’t calculate any output for those points. This is why our output is 2*2=4 rows/columns smaller than the input.pad
parameter of the convolutional layer in Lasagne). E.g. if you add 2 rows/cols of zeros around the image, the output size will be (28+4)-4=28. So in case of padding, the output size is input_size + 2*padding - (filter_size -1)
.stride=2
, which means that you move the filter in steps of 2 pixels. Then, the expression becomes ((input_size + 2*padding - filter_size)/stride) +1
.In your case, the full calculations are:
# name size parameters--- -------- ------------------------- ------------------------ 0 input 1x28x28 0 1 conv2d1 (28-(5-1))=24 -> 32x24x24 (5*5*1+1)*32 = 832 2 maxpool1 32x12x12 0 3 conv2d2 (12-(3-1))=10 -> 32x10x10 (3*3*32+1)*32 = 9'248 4 maxpool2 32x5x5 0 5 dense 256 (32*5*5+1)*256 = 205'056 6 output 10 (256+1)*10 = 2'570
So in your network, you have a total of 832 + 9’248 + 205’056 + 2’570 = 217’706 learnable parameters, which is exactly what Lasagne reports.
Installing CuDNN just involves placing the files in the CUDA directory. If you have specified the routes and the CuDNN option correctly while installing caffe it will be compiled with CuDNN.
You can check that using cmake
. Create a directory caffe/build
and run cmake ..
from there. If the configuration is correct you will see these lines:
-- Found cuDNN (include: /usr/local/cuda-7.0/include, library: /usr/local/cuda-7.0/lib64/libcudnn.so)-- NVIDIA CUDA:-- Target GPU(s) : Auto-- GPU arch(s) : sm_30-- cuDNN : Yes
If everything is correct just run the make
orders to install caffe from there.
if you use Keras and TensorFlow as Backend then you can try the following example. It calculates the FLOPs for the MobileNet.
import tensorflow as tfimport keras.backend as Kfrom keras.applications.mobilenet import MobileNetrun_meta = tf.RunMetadata()with tf.Session(graph=tf.Graph()) as sess: K.set_session(sess) net = MobileNet(alpha=.75, input_tensor=tf.placeholder('float32', shape=(1,32,32,3))) opts = tf.profiler.ProfileOptionBuilder.float_operation() flops = tf.profiler.profile(sess.graph, run_meta=run_meta, cmd='op', options=opts) opts = tf.profiler.ProfileOptionBuilder.trainable_variables_parameter() params = tf.profiler.profile(sess.graph, run_meta=run_meta, cmd='op', options=opts) print("{:,} --- {:,}".format(flops.total_float_ops, params.total_parameters))
All you have to do is define a function for that, using keras backend functions for calculations. The function must take the true values and the model predicted values.
Now, since I’m not sure about what are g, q, x an y in your function, I’ll just create a basic example here without caring about what it means or whether it’s an actual useful function:
import keras.backend as Kdef customLoss(yTrue,yPred): return K.sum(K.log(yTrue) - K.log(yPred))
All backend functions can be seen here: https://keras.io/backend/#backend-functions
After that, compile your model using that function instead of a regular one:
model.compile(loss=customLoss, optimizer = .....)
I want to explain with picture from C3D.
In a nutshell, convolutional direction & output shape is important!
↑↑↑↑↑ 1D Convolutions – Basic ↑↑↑↑↑
import tensorflow as tfimport numpy as npsess = tf.Session()ones_1d = np.ones(5)weight_1d = np.ones(3)strides_1d = 1in_1d = tf.constant(ones_1d, dtype=tf.float32)filter_1d = tf.constant(weight_1d, dtype=tf.float32)in_width = int(in_1d.shape[0])filter_width = int(filter_1d.shape[0])input_1d = tf.reshape(in_1d, [1, in_width, 1])kernel_1d = tf.reshape(filter_1d, [filter_width, 1, 1])output_1d = tf.squeeze(tf.nn.conv1d(input_1d, kernel_1d, strides_1d, padding='SAME'))print sess.run(output_1d)
↑↑↑↑↑ 2D Convolutions – Basic ↑↑↑↑↑
ones_2d = np.ones((5,5))weight_2d = np.ones((3,3))strides_2d = [1, 1, 1, 1]in_2d = tf.constant(ones_2d, dtype=tf.float32)filter_2d = tf.constant(weight_2d, dtype=tf.float32)in_width = int(in_2d.shape[0])in_height = int(in_2d.shape[1])filter_width = int(filter_2d.shape[0])filter_height = int(filter_2d.shape[1])input_2d = tf.reshape(in_2d, [1, in_height, in_width, 1])kernel_2d = tf.reshape(filter_2d, [filter_height, filter_width, 1, 1])output_2d = tf.squeeze(tf.nn.conv2d(input_2d, kernel_2d, strides=strides_2d, padding='SAME'))print sess.run(output_2d)
↑↑↑↑↑ 3D Convolutions – Basic ↑↑↑↑↑
ones_3d = np.ones((5,5,5))weight_3d = np.ones((3,3,3))strides_3d = [1, 1, 1, 1, 1]in_3d = tf.constant(ones_3d, dtype=tf.float32)filter_3d = tf.constant(weight_3d, dtype=tf.float32)in_width = int(in_3d.shape[0])in_height = int(in_3d.shape[1])in_depth = int(in_3d.shape[2])filter_width = int(filter_3d.shape[0])filter_height = int(filter_3d.shape[1])filter_depth = int(filter_3d.shape[2])input_3d = tf.reshape(in_3d, [1, in_depth, in_height, in_depth, 1])kernel_3d = tf.reshape(filter_3d, [filter_depth, filter_height, filter_width, 1, 1])output_3d = tf.squeeze(tf.nn.conv3d(input_3d, kernel_3d, strides=strides_3d, padding='SAME'))print sess.run(output_3d)
↑↑↑↑↑ 2D Convolutions with 3D input – LeNet, VGG, …, ↑↑↑↑↑
in_channels = 32 # 3 for RGB, 32, 64, 128, ... ones_3d = np.ones((5,5,in_channels)) # input is 3d, in_channels = 32# filter must have 3d-shpae with in_channelsweight_3d = np.ones((3,3,in_channels)) strides_2d = [1, 1, 1, 1]in_3d = tf.constant(ones_3d, dtype=tf.float32)filter_3d = tf.constant(weight_3d, dtype=tf.float32)in_width = int(in_3d.shape[0])in_height = int(in_3d.shape[1])filter_width = int(filter_3d.shape[0])filter_height = int(filter_3d.shape[1])input_3d = tf.reshape(in_3d, [1, in_height, in_width, in_channels])kernel_3d = tf.reshape(filter_3d, [filter_height, filter_width, in_channels, 1])output_2d = tf.squeeze(tf.nn.conv2d(input_3d, kernel_3d, strides=strides_2d, padding='SAME'))print sess.run(output_2d)
in_channels = 32 # 3 for RGB, 32, 64, 128, ... out_channels = 64 # 128, 256, ...ones_3d = np.ones((5,5,in_channels)) # input is 3d, in_channels = 32# filter must have 3d-shpae x number of filters = 4Dweight_4d = np.ones((3,3,in_channels, out_channels))strides_2d = [1, 1, 1, 1]in_3d = tf.constant(ones_3d, dtype=tf.float32)filter_4d = tf.constant(weight_4d, dtype=tf.float32)in_width = int(in_3d.shape[0])in_height = int(in_3d.shape[1])filter_width = int(filter_4d.shape[0])filter_height = int(filter_4d.shape[1])input_3d = tf.reshape(in_3d, [1, in_height, in_width, in_channels])kernel_4d = tf.reshape(filter_4d, [filter_height, filter_width, in_channels, out_channels])#output stacked shape is 3D = 2D x N matrixoutput_3d = tf.nn.conv2d(input_3d, kernel_4d, strides=strides_2d, padding='SAME')print sess.run(output_3d)
↑↑↑↑↑ Bonus 1×1 conv in CNN – GoogLeNet, …, ↑↑↑↑↑
in_channels = 32 # 3 for RGB, 32, 64, 128, ... out_channels = 64 # 128, 256, ...ones_3d = np.ones((1,1,in_channels)) # input is 3d, in_channels = 32# filter must have 3d-shpae x number of filters = 4Dweight_4d = np.ones((3,3,in_channels, out_channels))strides_2d = [1, 1, 1, 1]in_3d = tf.constant(ones_3d, dtype=tf.float32)filter_4d = tf.constant(weight_4d, dtype=tf.float32)in_width = int(in_3d.shape[0])in_height = int(in_3d.shape[1])filter_width = int(filter_4d.shape[0])filter_height = int(filter_4d.shape[1])input_3d = tf.reshape(in_3d, [1, in_height, in_width, in_channels])kernel_4d = tf.reshape(filter_4d, [filter_height, filter_width, in_channels, out_channels])#output stacked shape is 3D = 2D x N matrixoutput_3d = tf.nn.conv2d(input_3d, kernel_4d, strides=strides_2d, padding='SAME')print sess.run(output_3d)
– Original Link : LINK
– The author: Martin Görner
– Twitter: @martin_gorner
– Google +: plus.google.com/+MartinGorne
↑↑↑↑↑ 1D Convolutions with 1D input ↑↑↑↑↑
↑↑↑↑↑ 1D Convolutions with 2D input ↑↑↑↑↑
in_channels = 32 # 3, 32, 64, 128, ... out_channels = 64 # 3, 32, 64, 128, ... ones_4d = np.ones((5,5,5,in_channels))weight_5d = np.ones((3,3,3,in_channels,out_channels))strides_3d = [1, 1, 1, 1, 1]in_4d = tf.constant(ones_4d, dtype=tf.float32)filter_5d = tf.constant(weight_5d, dtype=tf.float32)in_width = int(in_4d.shape[0])in_height = int(in_4d.shape[1])in_depth = int(in_4d.shape[2])filter_width = int(filter_5d.shape[0])filter_height = int(filter_5d.shape[1])filter_depth = int(filter_5d.shape[2])input_4d = tf.reshape(in_4d, [1, in_depth, in_height, in_depth, in_channels])kernel_5d = tf.reshape(filter_5d, [filter_depth, filter_height, filter_width, in_channels, out_channels])output_4d = tf.nn.conv3d(input_4d, kernel_5d, strides=strides_3d, padding='SAME')print sess.run(output_4d)sess.close()
Let’s start with the terms. Remember that the output of the convolutional layer is a 4-rank tensor [B, H, W, C]
, where B
is the batch size, (H, W)
is the feature map size, C
is the number of channels. An index (x, y)
where 0 <= x < H
and 0 <= y < W
is a spatial location.
Now, here’s how the batchnorm is applied in a usual way (in pseudo-code):
# t is the incoming tensor of shape [B, H, W, C]# mean and stddev are computed along 0 axis and have shape [H, W, C]mean = mean(t, axis=0)stddev = stddev(t, axis=0)for i in 0..B-1: out[i,:,:,:] = norm(t[i,:,:,:], mean, stddev)
Basically, it computes H*W*C
means and H*W*C
standard deviations across B
elements. You may notice that different elements at different spatial locations have their own mean and variance and gather only B
values.
This way is totally possible. But the convolutional layer has a special property: filter weights are shared across the input image (you can read it in detail in this post). That’s why it’s reasonable to normalize the output in the same way, so that each output value takes the mean and variance of B*H*W
values, at different locations.
Here’s how the code looks like in this case (again pseudo-code):
# t is still the incoming tensor of shape [B, H, W, C]# but mean and stddev are computed along (0, 1, 2) axes and have just [C] shapemean = mean(t, axis=(0, 1, 2))stddev = stddev(t, axis=(0, 1, 2))for i in 0..B-1, x in 0..H-1, y in 0..W-1: out[i,x,y,:] = norm(t[i,x,y,:], mean, stddev)
In total, there are only C
means and standard deviations and each one of them is computed over B*H*W
values. That’s what they mean when they say “effective mini-batch”: the difference between the two is only in axis selection (or equivalently “mini-batch selection”).
there are several more ways to do batch-norm in TensorFlow:
tf.nn.batch_normalization
is a low-level op. The caller is responsible to handle mean
and variance
tensors themselves.tf.nn.fused_batch_norm
is another low-level op, similar to the previous one. The difference is that it’s optimized for 4D input tensors, which is the usual case in convolutional neural networks. tf.nn.batch_normalization
accepts tensors of any rank greater than 1.tf.layers.batch_normalization
is a high-level wrapper over the previous ops. The biggest difference is that it takes care of creating and managing the running mean and variance tensors, and calls a fast fused op when possible. Usually, this should be the default choice for you.tf.contrib.layers.batch_norm
is the early implementation of batch norm, before it’s graduated to the core API (i.e., tf.layers
). The use of it is not recommended because it may be dropped in the future releases.tf.nn.batch_norm_with_global_normalization
is another deprecated op. Currently, delegates the call to tf.nn.batch_normalization
, but likely to be dropped in the future.keras.layers.BatchNormalization
, which in case of tensorflow backend invokes tf.nn.batch_normalization
.Batch Normalization is just another layer, so you can use it as such to create your desired network architecture.
The general use case is to use BN between the linear and non-linear layers in your network, because it normalizes the input to your activation function, so that you’re centered in the linear section of the activation function (such as Sigmoid). There’s a small discussion of it here
In your case above, this might look like:
# import BatchNormalizationfrom keras.layers.normalization import BatchNormalization# instantiate modelmodel = Sequential()# we can think of this chunk as the input layermodel.add(Dense(64, input_dim=14, init='uniform'))model.add(BatchNormalization())model.add(Activation('tanh'))model.add(Dropout(0.5))# we can think of this chunk as the hidden layer model.add(Dense(64, init='uniform'))model.add(BatchNormalization())model.add(Activation('tanh'))model.add(Dropout(0.5))# we can think of this chunk as the output layermodel.add(Dense(2, init='uniform'))model.add(BatchNormalization())model.add(Activation('softmax'))# setting up the optimization of our weights sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)model.compile(loss='binary_crossentropy', optimizer=sgd)# running the fittingmodel.fit(X_train, y_train, nb_epoch=20, batch_size=16, show_accuracy=True, validation_split=0.2, verbose = 2)
This a classic use for cv2.dilate()
. Essentially when you want to connect items together, you can dilate them to join multiple items into a single item. Here’s a simple approach
import cv2import numpy as npimage = cv2.imread('test.png')gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)blur = cv2.GaussianBlur(gray, (7,7), 0)thresh = cv2.threshold(blur, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (5,5))dilate = cv2.dilate(thresh, kernel, iterations=4)cnts = cv2.findContours(dilate, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)cnts = cnts[0] if len(cnts) == 2 else cnts[1]for c in cnts: x,y,w,h = cv2.boundingRect(c) cv2.rectangle(image, (x, y), (x + w, y + h), (36,255,12), 2)cv2.imshow('th', thresh)cv2.imshow('dilated', dilate)cv2.imshow('image', image)cv2.waitKey()
Here’s where the magic happens. We can assume that a paragraph is a section of words that are close together, to achieve this we dilate to connect adjacent words.
1. Block Notation(plain, flow-style, scalar): Newlines become spaces and extra newlines after the block are removed
---# Note: It has 1 new line after the stringcontent: Arbitrary free text over multiple lines stopping after indentation changes......
Equivalent JSON
{ "content": "Arbitrary free text over multiple lines stopping after indentation changes..."}
2. Literal Block Scalar: A Literal Block Scalar | will include the newlines and any trailing spaces. but removes extra
newlines after the block.
---# After string we have 2 spaces and 2 new linescontent1: | Arbitrary free text over "multiple lines" stopping after indentation changes... ...
Equivalent JSON
{ "content1": "Arbitrary free textnover "multiple lines" stoppingnafter indentation changes... n"}
3. + indicator with Literal Block Scalar: keep extra newlines after block
---# After string we have 2 new linesplain: |+ This unquoted scalar spans many lines....
Equivalent JSON
{ "plain": "This unquoted scalarnspans many lines.nnn"}
4. – indicator with Literal Block Scalar: – means that the newline at the end of the string is removed.
---# After string we have 2 new linesplain: |- This unquoted scalar spans many lines....
Equivalent JSON
{ "plain": "This unquoted scalarnspans many lines."}
5. Folded Block Scalar(>):
will fold newlines to spaces and but removes extra newlines after the block.
---folded_newlines: > this is really a single line of text despite appearances...
Equivalent JSON
{ "fold_newlines": "this is really a single line of text despite appearancesn"}
It looks like you pip installed imutils using python2’s pip and so that’s why python3 can’t import it. You might try: sudo pip3 install imutils
Python 2:
sudo pip install imutils
Python 3:
sudo pip3 install imutils
Even after installing if it is showing error then use this :
sudo pip3 install --upgrade imutils
Answer: You can use PdfStamper for this purpose.
PdfReader reader = new PdfReader(src);PdfStamper stamper = new PdfStamper(reader, new FileOutputStream(dest));stamper.insertPage(reader.getNumberOfPages() + 1, reader.getPageSizeWithRotation(1));stamper.close();reader.close();
If src
refers to a document with 100 pages, the code above will add an extra blank 101st page, using the same page size as the first page.
I tried this one. This script runs a python file with the argument in Java. It also logs about which line, your program is executing.
Hope this Helps.
import java.io.BufferedReader; import java.io.IOException; import java.io.InputStreamReader; import java.io.Reader; public class Test { public static void main(String... args) throws IOException { ProcessBuilder pb = new ProcessBuilder("python","samples/test/table_cv.py","1.pdf"); pb.redirectErrorStream(true); Process proc = pb.start(); Reader reader = new InputStreamReader(proc.getInputStream()); BufferedReader bf = new BufferedReader(reader); String s; while ((s = bf.readLine()) != null) { System.out.println(s); } } }