TJ Machine Learning Club

Technical Resources

These are resources to consult if you're looking to supplement knowledge gained from club lectures.

• Coursera Courses

  • Machine Learning: This is a great course to get started with Machine Learning. You'll familiarize yourself with terminology and learn about traditional ML models. (The exercises are in a language called MATLAB, but this is not usually used nowadays for ML applications. It may be worth doing the exercises in Python, or just watching the videos.)

  • deeplearning.ai: This five course series is a great way to learn about neural networks and deep learning. You'll learn about neural networks, convolutional neural networks, recurrent neural networks. It will also give you hands-on experience with deep learning frameworks such as Keras and TensorFlow. Although it requires money to access the exercises and quizzes, you can audit the course and have access to the lectures. This course is great if you do not yet have a strong understanding of math concepts such as gradients and partial derivative because the instructor teaches them along the way.

  • Reinforcement Learning Specialization: This series of courses by the University of Alberta teaches you about reinforcement learning. You can audit the courses and have access to all the quizzes and videos.

• Online University Courses

  We recommend reading the lecture notes or watching the videos of the CS229 and CS231n courses. CS229 takes a more mathematical look at standard machine learning methods, while CS231n focuses on deep learning algorithms for visual processing. The rigorous lecture notes for CS229 are especially helpful.

  If you're interested in reinforcement learning, we recommend viewing the CS234 course notes, slides, or videos. If you want to learn more about generative models, slides and notes are available from the CS236 course, but no lecture recordings are available. Other interesting courses include CS224n, which covers natural language processing, CS228: Probabilistic Graphical Models, and CS330: Deep Multi-Task and Meta Learning.

  If some materials are unavailable because a course is currently in session, look for archived versions of previous offerings. Most courses should also have lecture videos from a previous year publicly available on YouTube.

• Books

  • Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems: This book (as the title suggests) gives you hands on experience with machine learning libraries. It is being used in TJ's new ML class.

  • Deep Learning: This book is the gold standard in universities teaching deep learning classes. It is available for free online. It does require some advanced math for the high-school level.

• Math Resources

• Khan Academy

• MIT OCW: MIT Open Courseware has high-quality lectures and problems for advanced math subjects.

• AOPS Calculus Textbook

• Stewart Multivariable Calculus Textbook: This is the textbook used for the TJ multivariable calculus class.

• Professor Leonard's YouTube Channel: This channel has full-length calculus and multivariable calculus playlists. It's great if you haven't taken multi yet.

• Other

• Spinning Up in Deep RL: This guide created by OpenAI can help you get started in both the theoretical and hands-on aspects of Reinforcement Learning.

Nontechnical Resources

Here are some resources if you are interested in learning more about the field of AI in a nontechnical manner:

• Artificial Intelligence Podcast: Created by Lex Fridman, a researcher in human-centered AI and deep learning at MIT, this podcast contains discussions with researchers and experts from the field of machine learning and other related fields. Notable guests on the podcast include: Ian Goodfellow (the creator of GANs), Yoshua Bengio (one of the fathers of neural networks), Oriol Vinyals (the creator of AlphaStar), Peter Norvig and Stuart Russell (authors of the AI textbook used at TJ), Donald Knuth, and Elon Musk. In addition to the link above, you can listen to episodes on any podcast provider.

• DeepMind: The Podcast: This podcast series gives you an inside look at Google DeepMind, a research organization that is at the bleeding edge of AI research. Throughout the eight episodes, host Hannah Fry speaks to researchers who are contributing to ongoing research projects at the organization. The podcast ends with an interview with the company's co-founder and CEO Demis Hassabis (we highly recommend watching this episode).

• AlphaGo - The Movie: This documentary takes you behind the scenes of perhaps the most successful AI project of the last decade: creating a machine capable of beating a professional player at the board game Go.

• Superintelligence: Paths, Dangers, Strategies: This book discusses the topic of AI safety and the potential pitfalls of advancing AI and machine learning without consideration. There is a copy in the SysLab library/closet (room 202).

• Human Compatible: AI and the Problem of Control: This book discusses ways to ensure that advanced AI ultimately remains under human control.

Computational Resources

Unfortunately, TJ has a severe lack of GPUs. When the Computer Systems lab was upgraded a few years ago, the aging CPUs were replaced with then-modern i5-4690s. However, the GPUs remain GT 730s. If you would like to run your code on TJ machines, we recommend you use the computer called zoidberg, which has modern GPUs. Simply ssh using the following commands:
ssh username@remote.tjhsst.edu
ssh zoidberg

All current students can access TJ syslab machines through ssh. Follow these instructions. If you would like to transfer data to the high performance computing cluster, take a look at this guide.

Installing Libraries on TJ Linux Machines

All the necessary dependencies (CUDA, cuDNN) have been installed for TensorFlow, Keras, and PyTorch on the Linux machines. Assuming the library you want hasn't already been installed, you'll need to install it with the --user flag. Use these commands on machines with GPUs for TensorFlow and Keras, respectively:
pip3 install tensorflow-gpu --user
pip3 install keras --user

Note that Keras requires a backend engine (TensorFlow is recommended).

For PyTorch, check the latest commands on the website. Select the proper Python and CUDA versions. You can check your python version by typing python and the CUDA version with nvcc --version.

If you are working on a standard Linux machine without a decent GPU, you'll need to install the CPU versions of these libraries. For TensorFlow, installation instructions are available here. For Pytorch, ensure you select "None" for the CUDA version. As always, you will need to add --user to all commands if you aren't working in a virtual environment, or don't have root.

Installing Libraries on Your Own Linux Machine

If you want to install these libraries on your own Linux computer, just remove the --user for the above commands, since you presumably have root. If you don't have a half-decent Nvidia GPU, running large deep learning models will take unreasonably long, and it is advisable you use the TJ GPU cluster. If you do have a GPU, make sure you have CUDA and cuDNN configured beforehand.

Libraries on Windows Machines

Most machine learning libraries are compatible with Windows, especially CPU versions. GPU-dependent deep learning libraries are often slower and less reliable on Windows, so use Linux if you can.

Machine Learning Club recommends using scikit-learn for simple machine learning. You can install it using:
pip3 install -U scikit-learn

If you are trying to run on TJ Windows machines, a bit of a workaround is required due to restrictions. Follow these instructions for scikit-learn.

Instructions for installing scikit-learn on TJHSST Windows Machines

If you can't get that to work, use our Instructions for ssh'ing into the syslab

Basic Vim and Nano Instructions

Python 2 or 3?

Machine Learning Club recommends Python 3 whenever possible. Almost all libraries have support for the latest version. Even though most support Python 2.7 as well, Python 2 support ends in 2020, so make the switch if you haven't already!