In a neural network, the output value of a neuron is almost always transformed in some way using a function. A trivial choice would be a linear transformation which can only scale the data. However, other transformations, like a step function allow for non-linear properties to be introduced. Activation functions can also help to standardize your data between layers. Some functions such as the sigmoid have the effect of “focusing” the area of interest on data. Extreme values are placed close together, while values near it’s point of inflection change more quickly with respect to small changes in the input. Similarly, these functions can take any real number and map all of them to a finite range such as [0, 1] which can have many advantages for downstream calculation. In this episode, we overview the concept and discuss a few reasons why you might select one function verse another. About the “Data Skeptic” Podcast
The Data Skeptic Podcast features interviews and discussion of topics related to data science, statistics, machine learning, artificial intelligence and the like, all from the perspective of applying critical thinking and the scientific method to evaluate the veracity of claims and efficacy of approaches.
This episode recaps the Microsoft Build Conference. Kyle recently attended and shares some thoughts on cloud, databases, cognitive services, and artificial intelligence. The episode includes interviews with Rohan Kumar and David Carmona. About the “Data Skeptic” Podcast
Max-pooling is a procedure in a neural network which has several benefits. It performs dimensionality reduction by taking a collection of neurons and reducing them to a single value for future layers to receive as input. It can also prevent overfitting, since it takes a large set of inputs and admits only one value, making it harder to memorize the input. In this episode, we discuss the intuitive interpretation of max-pooling and why it’s more common than mean-pooling or (theoretically) quartile-pooling. About the “Data Skeptic” Podcast
This episode is an interview with Tinghui Zhou. In the recent paper “Unsupervised Learning of Depth and Ego-motion from Video“, Tinghui and collaborators propose a deep learning architecture which is able to learn depth and pose information from unlabeled videos. We discuss details of this project and its applications. About the “Data Skeptic” Podcast
CNNs are characterized by their use of a group of neurons typically referred to as a filter or kernel. In image recognition, this kernel is repeated over the entire image. In this way, CNNs may achieve the property of translational invariance – once trained to recognize certain things, changing the position of that thing in an image should not disrupt the CNN’s ability to recognize it. In this episode, we discuss a few high-level details of this important architecture. About the “Data Skeptic” Podcast