(Continued) Week2- Neural Networks Basics #100DaysOfMLCode Challenge

*This post is part of my #100DaysOfMLCode challenge. Learning about Machine Learning for 100days continuously. I have taken images from my specialization course which I have taken on Coursera.

*please ignore my writing skills. and also I didn’t organize the information well. This Post is about what I have understood from the videos which I have watched.

Coursera’s Deep Learning Specialization course homepage: https://www.coursera.org/learn/neural-networks-deep-learning#

---

---

Python and Vectorization

Vectorization

• Vectorization is basically a method of getting rid of explicit for-loops.
• In the deep learning era, code runs for relatively large datasets. So our code needs to perform well. Vectorization plays a key role here.
•  vectorized code runs faster than non-vectorized code.
• SIMD(Single Instruction Multiple Data)

---

---

More Vectorization Examples

• By avoiding explicit for loops, your code will run significantly faster.
• The rule of thumb to keep in mind is “whenever you are programming neural network or logistic regression just avoid the explicit for-loops whenever possible.”
• Whenever you want to compute Vector ‘u’ as a product of matrix A and another vector ‘v’, then
•  definition of matrix multiplication
•  in the non-vectorized version, we need to have two for loops.
• this is the vectorized version, this will eliminate the use of two- for loops.
• suppose you need to apply the exponential operation on every element of a vector/matrix.
•  As you can see , there is a non-vectorized implementation on the left side. Where as right side contains Vectorized implementation. Vectorized implementation will allows us to eliminate the explicit for-loop.
• using numpy functions we can able to apply the similar operation to all the elements of a vector without the use of explicit for loop.
•  we have eliminated the second for-loop from the algorithm. Instead of a for-loop we have used vector.

---

Vectorizing Logistic Regression

• Vectorizing Logistic Regression means implementing Logistic Regression without a single for-loop.
• predictions can be calculated in a single step rather than for each individual training element.
• with one line of code, we can able to calculate capital Z.

---

---

Vectorizing Logistic Regression’s Gradient Output

• we have done forward propagation and backward propagation by computing the derivatives and predictions on all the training examples without using a for-loop.
• Gradient Descent is updated as following.
• we may need an explicit for-loop whenever we need to iterate it for multiple times.
• We can able to do a single iteration of for-loop without using a for-loop.
• Broadcasting a technique through which certain parts of the code can be made more efficient and faster.

---

Broadcasting in Python

•  Can you calculate the percentage of calories from carbs, protein, and fats without the use of explicit for-loop.
•  axis=0 represents the adding the sum of the elements vertically.
•  this is an example of broadcasting in python.
• basic process of broadcasting.
• General principle of broadcasting.

---

A Note on python/numpy vectors

•  whenever you create a vector, don’t use the “rank 1” type vectors.
•  if you are not sure about the dimensions then you can use assert statement.
• Either use column vector or row vector.

---

Logistic Regression Cost Function

• single equation depicting Logistic Regression in two forms.

---

Python Basics with Numpy

1. In this particular Programming exercise we will learn
2. How to use Numpy
3. Implement some basic core deep learning functions such as the softmax, sigmoid, dsigmoid, etc.
4. Learn how to handle data by normalizing inputs and reshaping images.
5. Recognize the importance of vectorization.
6. Understand how python broadcasting works.
7. I have learnt about building basic functions with numpy.
8. The data structures we use in numpy to represent the shapes ( vectors, matrices, etc) are called numpy arrays.
9. Implemented logistic sigmoid function using numpy.
10. Implemented sigmoid gradient using numpy.
11. Two common numpy functions used in deep learning are np.shape and np.reshape.
12. Learnt how to reshape the numpy arrays.
13. Implemented broadcasting and softmax function.
14. How vectorization plays an important role in computation of vectors and matrices.

• Vectorization plays an important role in deep learning. It provides computational efficiency and clarity.
• Implemented the basicsigmoid, sigmoid, sigmoid_derivative, image2vector, normalize rows, softmax, L1 function and L2 function.

---

Logistic Regression with a Neural Network Mindset

1. In this Programming assignment we are going to work with logistic regression in a way that builds intuition relevant to neural networks
2. By doing this assignment you will get to know about building the general architecture of a learning algorithm
3. You will get to know about initializing parameters, calculating the cost function and its gradient. Using an optimization algorithm as well

• numpy is the fundamental package for scientific computing with Python.
• h5py is a common package to interact with a dataset that is stored on an H5 file.
• matplotlib is a famous library to plot graphs in Python.
• PIL and scipy are used here to test your model with your own picture at the end.

8. Many software bugs in deep learning come from having matrix/vector dimensions that don’t fit. If you can keep your matrix/vector dimensions straight you will go a long way toward eliminating many bugs.

Common steps for pre-processing a new dataset are:

1. Figure out the dimensions and shapes of the problem (m_train, m_test, num_px, …)
2. Reshape the datasets such that each example is now a vector of size (num_px * num_px * 3, 1)
3. “Standardize” the data

The main steps for building a Neural Network are:

• Define the model structure(such as number of input features)
• Initialize the model’s parameters.
• LOOP( calculate current loss (forward propagation), Calculate current gradient (backward propagation), update parameters(gradient descent))

Implemented the following functions by doing this exercise.

1. Initializing the values of w,b.
2. Optimize the loss iteratively to learn the parameters (w,b)
3. Computing the cost and its gradient.
4. Updating the parameters using gradient descent.
5. Used the learned (w,b) to predict the labels for a given set of examples.

• Different learning rates gives different costs.
• In deep learning it is recommended to chose the learning rate which minimizes the overall cost. But if the model overfits , use other techniques to reduce overfitting.

---

 

Advertisement