Ever wanted to understand and build your own neural network?
Now is the time! Saddle up, power up your laptop, and let's get building!
FYI - (I'm coding this post in Python because it's the go-to for machine learning these days. Let me know if you want it in another language)
Let's get going: Get yourself over here and open: Google Colab: on your browser.
Crack open a new workbook and write the following lines :
import tensorflow as tf
print("TensorFlow version is:", tf.__version__)Did it work? Yes! wicked. Let's crack on then.
If not, check your spelling, and if you still have no luck drop me a comment ;)
Next, we load the MNIST dataset...
What the MNIST are you talking about?!
If you've read about AI, then I have no doubt you'll come across the MNIST data set. MNIST is like the "Hello world!" of machine learning. (Now I can't take credit for this analogy, but I can't remember where I read it, so props to the author that did).
A quick definition for funsies:
The MNIST dataset is a large database of handwritten digits that is commonly used for training various image processing systems. It contains a collection of 70,000, 28 x 28 images of handwritten digits from 0 to 9. The MNIST dataset is often used for benchmarking machine learning algorithms and contains 60,000 training images and 10,000 testing images.
Now sadly it doesn't look quite as elegant as this on the dataset. The real training data is a bit basic and consists of greyscales. You can see it with any Google search, so I made this one with the help of my trusty AI :D Pretty isn't it! :)
Chuck this line in your notebook:
mnist = tf.keras.datasets.mnistIf at this point you're staring and drooling. Never fear, explanations are here, let's dig in.
Being John Malkovich (the competent programmer)
One of the tricks of being a competent programmer is understanding what you're copying and pasting.
As many of us got better thanks to sites like Stack Overflow - (despite all the unbelievably aggressive elitism and roaming assholes) - at some point you need to transition over to original thinking and original coding.
To be honest, that's where the joy of coding is for me, and where I find that elusive state of 'flow'.
When I'm into the code and my mind is totally focused on the structures I'm creating and the objects I'm passing around, it's a wonderful feeling. You just don't get that from googling and copying and pasting solutions out of stack overflow with the hope that one will stick and work.
To get to that state needs a deeper level of understanding, and whilst that is more difficult to develop, I assure you the rewards are immense when you get there. So bear that in mind as you learn to code, taking your time and deep diving is almost always worth it.
So let's dig a bit deep, and deep dive into this sucker to understand it.
mnist = tf.keras.datasets.mnist
What the hell does that mean? Understanding starts with the API.
In case you don't know, an API is a set of tools you can command within the application you're using. In this example, we are using the TensorFlow application which we've defined as tf (See the first line of code)-> so the first next step is to know what the Keras means. So let's check out the API in the link below:
Head here with your browser (but obviously keep this window open and maybe just refresh a few more times or log on from your PC or phone and your partner's phone, and your friend's phone, whatever... I'm sure they won't mind. I haven't worked out how to monetize this site yet but I'm sure if I do it will be something to do with how many hits I get :D)
If you follow this rabbit hole of tf -> keras -> datasets -> minsts you'll come across several modules. Keras is the first one we get to :
What's Keras then?
Keras is the high-level API of the TensorFlow platform. It provides essential abstractions and building blocks for developing. Right, don't worry too much about that right now. Let's move on.
Then we head to the datasets module, not much to see here.
In the MNIST module - this module has been built to deal specifically with the MNIST dataset. Clearly, they were expecting us. AI in action :D.
Just in case you're not sure what a module is, In programming, a module is a self-contained unit of code that can be reused and imported into other programs.l It is a way to organize code into separate, independent components that can be easily maintained and tested.
{Side note: Alongside the mnist module, you'll see a couple of other things like IMDb and Reuters. I'm sure you've all heard about those things, so out of curiosity check out IMDB - you'll see a couple of functions there, one of which retrieves an IMDB dataset. Or perhaps you're more into your stocks game in which Reuters will tickle your pickle.
Either way. How cool is that! IMDB or stocks are probably way more interesting than some MNIST datasets... but perhaps we will go to that once we've figured out how to do the MINST stuff. Crawl then sprint.}
Focus please, back to the neural network
So here we are, and the only thing we have available to us and clearly are supposed to use is the load_data function. Load Data API Document
Load Data, So what does it do?
Under returns, (which logically means what it will send back our way when we run the command) it is telling us it returns a tuple of NumPy arrays: That may cause a bit of a brain fuzz/slash dump. Stick with it, things will get clearer.
Let's just go ahead and run that command and see if there is any output:
mnist.load_data()Okay, so a bunch of brackets and a bunch of zeroes with the occasional word array thrown in.
Before we dive any further, it's probably a good time to talk about what tensors are
Why is it called a Tensor flow, what does that even mean!?
Good question! It's because it's software designed to work on tensors.
Oh okay, what's a Tensor then: an N-dimensional array that represents data. Okay, great, still not clear.
Let's pretend we have a tensor robot in front of us, let's call it the Tensor Operations Support Robot (or TosBot) and have a conversation with it:
TosBot Chat Excerpt:
Hello Tensor, what are you?
'I'm a data structure'
Okay, what kind?
Well, that depends on how complex I am
Okay, still not clear, elaborate, please?
Well, I'm known as an N-dimensional array, therefore
If N is 0, I just store a single value.
If you remember from school that if we have a single value it's known as a scalar. Like, the temperature in the room.
Condescending.
if N is 1, then you have two values, it's a vector. Like throwing a ball has a direction and a speed.
if N is 2, then you have three values, like a matrix, so that could point out a position in say a 3D space with X,Y and Z co-ordinates, like a cube of data.
Okay
if N is 3, then you have an additional matrix attached, giving an additional depth.
An example might be a tv screen, you have your X and Y co-ordinate, but when you get there, you get a pixel broken down into a further matrix of red, green, and blue...)
Erm..
if N is 4 ...
Okay stop my head hurts now. Awesome, we have now evolved. Let's continue building our neural network!
So now we can see that the API describes that the return values from the MNIST load_data function are as follows:
x_train: uint8 NumPy array of grayscale image data with shapes (60000, 28, 28), containing the training data. Pixel values range from 0 to 255.
y_train: uint8 NumPy array of digit labels (integers in range 0-9) with shape (60000,) for the training data.
x_test: uint8 NumPy array of grayscale image data with shapes (10000, 28, 28), containing the test data. Pixel values range from 0 to 255.
y_test: uint8 NumPy array of digit labels (integers in range 0-9) with shape (10000,) for the test data.
(In case you are wondering, uint8 just means unsigned integer - it's stored with 8 bits and can store values between 0 to 255. A perfect size of memory for our 255 possible shades of gray, what a co-incidence :D ? or Jo-incidence as I fondly remember Matthew Perry - such a sad loss. Drugs are bad)
So in the API it gives us some an example of how to load the data and then test it:
This line hopefully now makes sense we are populating our collection (or tuple) of NumPy arrays:
(x_train, y_train), (x_test, y_test) = mnist.load_data()
The astute among you as well as those who can read will have noticed some chat about 'assert' afterwards:
assert x_train.shape == (60000, 28, 28)
assert x_test.shape == (10000, 28, 28)
assert y_train.shape == (60000,)
assert y_test.shape == (10000,)
In Python, the assert statement is a built-in construct that allows you to test assumptions about your code.
The assert statement takes an expression and an optional error message.
If the expression evaluates to False, the program will raise an AssertionError with the specified error message. Cool, so let's have some fun with a simple example of this in use.
Run the following line of code:
x = 5
assert x == 5, "x should be 5"Great stuff, nothing happened right... so what if we do this:
x = 4
assert x == 5, "x should be 5"Boom, and there it is. Our first test together. That makes sense right?
AssertionError: x should be 5So from the suggested code below what we can see is that we are testing the collection of NumPy arrays that are filled with data as we would expect them to be with the collection of assert checks.
assert x_train.shape == (60000, 28, 28)
assert x_test.shape == (10000, 28, 28)
assert y_train.shape == (60000,)
assert y_test.shape == (10000,)Now if you run that it will just (hopefully) remain quiet as everything fits as one would expect. However, I'm a visual person. I want to see what is inside some of these data collections.
Let's Get Visual!
So let's grab some of our test data and just print it out - see what it looks like.
I've chosen the test data because I'm expecting this to sort of break things a bit, and as a choice between printing out a splurge of 10,000 and 60,000 lines of numbers, I'll go lower :)
for row in x_test:
print(row)So you probably saw, like I did, a couple of errors popping up as it was trying to throw out at light speed many many lines of digits.
Let's grab a line:
[ 0 0 0 0 0 15 219 252 252 106 47 47 78 161 203 253 252 252 235 102 0 0 0 0 0 0 0 0]So here we can see the 0's represent black and the 253 represents the high white point.
A single line horizontally from some arbitrary point in our image.
Let's take this visualization to the next step.
I want to see the complete map of a single character.
Before I thought about it, I just created a simple for loop assuming that the X and Y would correspond to a single digit, so with my nested for loop below I tried to print out the first test digit:
x = 29
y = 29
for row_x in x_test:
if x==0:
break
x-=1
print(row_x)
for row_y in y_test:
if y==0:
break
y-=1
print(row_y)
.. Output yikes
Woah, I got a stream of 28 - sort of - digits in a blur.
Whilst this did print out something cool, it's not what I was expecting.
Something has gone wrong!!
So let's take a step back and have another look at the data structure:
x_test.shape == (10000, 28, 28)
y_test.shape == (10000,)Ah, my bad. The y doesn't hold the vertical axis at all.
If you run a quick for loop looking at the y values - it gives you the 'answer' to what is held in the x test shape.
So for example, the first image should be a 7. Okay, let's try that again.
y = 1
for row_y in y_test:
if y==0:
break
y-=1
print(row_y)
output -> 7
So let's go back to the plan.
After a rethink, we know there are 10,000 records in our x_test data, and each record has a 28 x 28 matrix - that's our digit.
So what we really need to do is just loop through one record of the data in the x_test and that should give us our digit.
x = 1
for data in x_test:
if x==0:
break
x-=1
print(data)
And voila!
Haha, How cool is that!
No one can be told what the Matrix is, they have to see it for themselves.
(If you're getting something similar to this but it looks a bit disjointed, it's probably because you've got a limit to the number of characters on screen, so just copy and paste into a notepad or VS code or something and with a bit of formatting so each line starts and ends in a '[' ']' you can enjoy this in all its glory! )
Now we are in a really good position to continue with building our neural network because we can actually see what we are showing our AI and we understand the data structure we are using.
See you tomorrow for Tensor Flow Tuesday where we will go into stage 2 of our neural network building! :D
Have a brilliant Saturday!! (Even though this blog is for Monday)
(Okay to be fair the day labeling/numbering is even confusing me now as the author - this is so typical of bad programming, the irony).
Comments