updated notes and implemented data generation for 10 digit recognition

SecondSaturday/Mnist10Digit.ipynb

@@ -0,0 +1,34 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 2",
+   "language": "python",
+   "name": "python2"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 2
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython2",
+   "version": "2.7.14"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}

SecondSaturday/Notes.md

@@ -54,7 +54,7 @@ eg: stock market temporal data / speech data/ image data
 
 Non-Parametric models : k-NN(K-nearest neighbor), Decision Trees, Random Forests (independent of parameters)
   -> very inaccurate because doesn't know much about the data
-Parametric Models: based on fixed set of parameters
+Parametric Models: based on fixed set of parameters,SVM
   -> more accurate coz the knows more about the parameters from the data
 
 

SecondSaturday/Workshop.md

@@ -12,8 +12,7 @@ language, keyboard layout
   Predict the word
 
 ## Model
-  Structured Output/HMM/ CNN?
-
+  Structured Output/HMM/CNN?
 
 # mnist hand-written digit database -> build application for recognizing full phone numbers(10 digit).
 

SecondSunday/Notes.md

@@ -0,0 +1,22 @@
+# Activation Function (AKA Transfer function)
+ In a neural network activation function adds non-linearity to it.
+ Types:
+ 1. Sigmoid(Logistic) (used mostly for output layer(looks like probability))
+ 2. RelU or Rectified Linear Unit (important discovery for NN - most-used for hidden layers)(not suitable for output layer if output is supposed to be probability) and leaky RelU with some slope on negative part
+ 3. tanH (Hyperbolic) (-1 - 1) or ArcTan (Tan Inverse -> maps to -Pi/2 - Pi/2)
+ 4. Linear(or Identity) layer (used for output layers(best for regression))
+ 5. Softmax (classification giving probability) (probability coz outputs add upto 1)
+ 6. SquareRoot
+ 7. Exponential
+ 8. Sine.
+ 9. Ramp
+ 10. Step (Binary)
+ 11. Unit Sum
+
+if the network computation is something that is multiplicative, use log as activation so that the sum becomes addition.
+
+Constraint Optimization: optimize in such a way that the output is constrained to some value.
+
+
+Steps => number of iteration of batches
+Epoch => number of iterations of going throught the entire dataset

SecondSunday/mnist_phone_number.py

@@ -0,0 +1,46 @@
+%matplotlib inline
+from tensorflow.examples.tutorials.mnist import input_data
+import numpy as np
+import matplotlib.pyplot as plt
+
+mnist = input_data.read_data_sets('./SecondSunday/mnist_data', one_hot=True)
+label_number = mnist.train.labels.argmax(axis=1)
+number_imgs = {str(i):mnist.train.images[np.argwhere(label_number == i).squeeze()] for i in range(10)}
+DATA_COUNT = 100
+phone_number_digits = np.random.randint(10**9,10**10,size=(DATA_COUNT,10))
+phone_number_digits = np.random.randint(10,size=(DATA_COUNT,10))
+phone_number_digits.astype(str)
+phone_number_strings = phone_number_digits.astype(str)
+
+def pick_img(num):
+    rand_idx = np.random.randint(number_imgs[num].shape[0])
+    img = number_imgs[num][rand_idx].reshape(28,28)
+    return img
+
+def create_phone_img(phon_no):
+    return np.hstack(tuple([pick_img(d) for d in phon_no]))
+
+def create_phone_images(phone_array):
+    phone_number_images = []
+    for phon_no in phone_array:
+        phone_number_images.append(create_phone_img(phon_no))
+    return np.array(phone_number_images).reshape(-1,28*280)
+
+phone_number_imgs = create_phone_images(phone_number_strings)
+
+from keras.models import Sequential
+from keras.layers import Dense, Activation
+
+model = Sequential([
+    Dense(32, input_shape=(7840,)),
+    Activation('relu'),
+    Dense(10),
+    Activation('linear'),
+])
+
+model.compile(optimizer='rmsprop',
+              loss='categorical_crossentropy',
+              metrics=['accuracy'])
+
+model.fit()
+# plt.imshow(phone_number_imgs[np.random.randint(phone_number_imgs.shape[0])])