Welcome to the Computer Vision capstone project in the AI Nanodegree program! In this project, you’ll combine your knowledge of computer vision techniques and deep learning to build and end-to-end facial keypoint recognition system. Facial keypoints include points around the eyes, nose, and mouth on any face and are used in many applications, from facial tracking to emotion recognition. Your completed code should be able to take in any image containing faces and identify the location of each face and their facial keypoints, as shown below.

The project will be broken up into a few main parts in one Python notebook:

Part 1 : Investigating OpenCV, pre-processing, and face detection

Part 2 : Training a Convolutional Neural Network (CNN) to detect facial keypoints

Part 3 : Putting parts 1 and 2 together to identify facial keypoints on any image!

You'll also be given optional exercises that allow you to extend this project so that it works on video and allows you to implement fun face filters in real-time!

All of the starting code and resources you'll need to compete this project are in a Github repo! Before you can get stared coding, you'll have to make sure that you have all the libraries and dependencies required to support this project.

This project requires GPU acceleration to run efficiently. Please refer to the Udacity instructions for setting up a GPU instance for this project, and refer to the project instructions in the classroom for setup. link for AIND students

You should follow the AWS instructions in your classroom for best results.

1. Clone the repository, and navigate to the downloaded folder.

git clone https://github.com/udacity/AIND-CV-FacialKeypoints.git
cd AIND-CV-FacialKeypoints

2. Create (and activate) a new environment with Python 3.5 and the numpy package.

   • Linux or Mac:

   conda create --name aind-cv python=3.5 numpy
   source activate aind-cv
   

   • Windows:

   conda create --name aind-cv python=3.5 numpy scipy
   activate aind-cv
   

3. Install/Update TensorFlow (for this project, you may use CPU only).

   • Option 1: To install TensorFlow with GPU support, follow the guide to install the necessary NVIDIA software on your system. If you are using the Udacity AMI, you can skip this step and only need to install the tensorflow-gpu package:

   pip install tensorflow-gpu==1.1.0
   

   • Option 2: To install TensorFlow with CPU support only:

   pip install tensorflow==1.1.0
   

4. Install/Update Keras.

pip install keras -U

5. Switch Keras backend to TensorFlow.

   • Linux or Mac:

   KERAS_BACKEND=tensorflow python -c "from keras import backend"
   

   • Windows:

   set KERAS_BACKEND=tensorflow
   python -c "from keras import backend"
   

6. Install a few required pip packages (including OpenCV).

pip install -r requirements.txt

All of the data you'll need to train a neural network is in the AIND-CV-FacialKeypoints repo, in the subdirectory data. In this folder are a zipped training and test set of data.

1. Navigate to the data directory

cd data

2. Unzip the training and test data (in that same location). If you are in Windows, you can download this data and unzip it by double-clicking the zipped files. In Mac, you can use the terminal commands below.

unzip training.zip
unzip test.zip

You should be left with two .csv files of the same name. You may delete the zipped files.

Troubleshooting: If you are having trouble unzipping this data, you can download that same training and test data on Kaggle.

Now, with that data unzipped, you should have everything you need!

1. Navigate back to the repo. (Also your source environment should still be activated at this point)

cd
cd AIND-CV-FacialKeypoints

2. Open the notebook and follow the instructions.

jupyter notebook CV_project.ipynb

NOTE: While some code has already been implemented to get you started, you will need to implement additional functionality to successfully answer all of the questions included in the notebook. Unless requested, do not modify code that has already been included.

Your project will be reviewed by a Udacity reviewer against the Computer Vision project rubric. Review this rubric thoroughly, and self-evaluate your project before submission. All criteria found in the rubric must meet specifications for you to pass.

When you are ready to submit your project, collect the following files and compress them into a single zip archive for upload:

• The CV_project.ipynb file with fully functional code. All code cells should be executed and displaying output, and all questions should be answered.
• An HTML or PDF export of the project notebook with the name report.html or report.pdf.
• Any additional images used for the project that were not supplied to you for the project.
• Please do not include the project data sets in the data/ folder. They are too big and only your executed notebook code and text will be evaluated.

Alternatively, your submission could consist of only the GitHub link to your repository.