Deep Learning
Deep Learning with R by Francois Chollet, 2018
Artificial Intelligence field, was born in the 1950s, defined as the effort to automate intellectual tasks normally performed by humans. AI field encompasses machine learning and deep learning, but also includes many more approaches that don't involve any learning. Human-level artificial intelligence could be achieved by having programmers handcraft a sufficiently large set of explicit rules for manipulating knowledge, known as symbolic AI. Although symbolic AI proved suitable to solve well-defined, logical problems, such as playing chess, it turned out to be intractable to figure out explicit rules for solving more complex, fuzzy problems, such as image classification, speech recognition, and language translation. A new approach arose to take symbolic AI's place: machine learning.
Machine Learning
Machine learning arises from this question: could a computer go beyond whatever we know how to order it to perform and learn on its own how to perform a specified task? Could a computer surprise us? Rather than programmers crafting data-processing rules by hand, could a computer automatically learn these rules by looking at data?
This question opens the door to a new programming paradigm.
In classical programming, the paradigm of symbolic AI, humans input rules (a program) and data to be processed according to these rules, and out come answers.
Rules --->
CLASSICAL PROGRAMMING --> Answers
Data --->
With machine learning, humans input data as well as the answers expected from the data, and out come the rules. These rules can then be applied to new data to produce original answers.
Data --->
MACHINE LEARNING --> Rules
Answers --->
Artificial Intelligence field, was born in the 1950s, defined as the effort to automate intellectual tasks normally performed by humans. AI field encompasses machine learning and deep learning, but also includes many more approaches that don't involve any learning. Human-level artificial intelligence could be achieved by having programmers handcraft a sufficiently large set of explicit rules for manipulating knowledge, known as symbolic AI. Although symbolic AI proved suitable to solve well-defined, logical problems, such as playing chess, it turned out to be intractable to figure out explicit rules for solving more complex, fuzzy problems, such as image classification, speech recognition, and language translation. A new approach arose to take symbolic AI's place: machine learning.
Machine Learning
Machine learning arises from this question: could a computer go beyond whatever we know how to order it to perform and learn on its own how to perform a specified task? Could a computer surprise us? Rather than programmers crafting data-processing rules by hand, could a computer automatically learn these rules by looking at data?
This question opens the door to a new programming paradigm.
In classical programming, the paradigm of symbolic AI, humans input rules (a program) and data to be processed according to these rules, and out come answers.
Rules --->
CLASSICAL PROGRAMMING --> Answers
Data --->
With machine learning, humans input data as well as the answers expected from the data, and out come the rules. These rules can then be applied to new data to produce original answers.
Data --->
MACHINE LEARNING --> Rules
Answers --->
A machine-learning system is trained rather than explicitly programmed. It's presented with many examples relevant to a task, and it finds statistical structure in these examples that eventually allows the system to come up with rules for automating the task. Machine learning is tightly related to mathematical statistics, but it differs from statistics in several important ways. Unlike statistics, machine learning tends to deal with large, complex datasets (such as a dataset of millions of images, each consisting of tens of thousands of pixels). As a result, machine learning, and especially deep learning, exhibits comparatively little mathematical theory, and is engineering oriented. It's a hands-on discipline in which ideas are proven empirically more often than theoretically.
Machine learning discovers rules to execute a data processing task, given examples of what's expected. So, to do machine learning, we need three things:
1. Input data points -- for instance, if the task is speech recognition, these data points could be sound files of people speaking. If the task is image tagging, they could be pictures.
2. Examples of the expected output --in a speech-recognition task, these could be human-generated transcripts of sound files. If an image task, expected outputs could be tags such as "dog", "cat," and so on.
3. A way to measure whether the algorithm is doing a good job -- this is necessary in order to determine the distance between the algorithm's current output and its expected output. the measurement is used as a feedback signal to adjust the way the algorithm works. This adjustment step is what we call learning.
A machine-learning model transforms its input data into meaningful outputs, a process that is 'learned' from exposure to known examples of inputs and outputs. Therefore, the central problem in machine learning and deep learning is to meaningfully transform data: in other words, to learn useful representations of the input data at hand -- representations that get us closer to the expected output -- to represent or encode data. For instance, a color image can be encoded in the RGB format or in the HSV format: these are two different representations of the same data. Machine learning is. technically: searching for useful representations of some input data, within a predefined space of possibilities, using guidance from a feedback signal. This simple idea allows for solving a remarkably broad range of intellectual tasks, from speech recognition to autonomous car driving.
Deep learning
Deep learning is a specific subfield of machine learning: a new take on learning representations from data that puts an emphasis on learning successive layers of increasingly meaningful representations. How many layers contribute to a model of the data is called the depth of the model (layered representations learning or hierarchical representations learning). Modern deep learning often involves tens or even hundreds of successive layers of representations -- and they're all learned automatically from exposure to training data. Meanwhile, other approaches to machine learning tend to focus on learning only one or two layers of representations of the data; hence, they're sometimes called shallow learning.
In deep learning, these layered representations are learned via models called neural networks, structured in literal layers stacked on top of each other. The term neural network is a reference to neurobiology, but although some of the central concepts in deep learning were developed in part by drawing inspiration from our understanding of the brain, deep-learning models are not models of the brain. There's no evidence that the brain implements anything like the learning machanisms used in modern deep-learning models. For our purposes, deep learning is a mathematical framework for learning representations from data.
The network transforms the digit image into representations that are increasingly different from the original image and increasingly informative about the final result. You can think of a deep network as a multistage information-distillation operation, where information goes through successive filters and comes out increasingly purified )that is, useful with regard to some task). So, deep learning is, technically: a multistage way to learn data representations. It's a simple idea -- but, as it turns out, very simple mechanisms, sufficiently scaled, can end up looking like magic.
Machine learning is about mapping inputs (such as images) to targets (such as the label "cat"), which is done by observing many examples of inputs and targets. Deep neural networks do this input-to-target mapping via a deep sequence of simple data transformations (layers) and that these data transformations are learned by exposure to examples.
The specification of what a layer does to its input data is stored in the layer's weights, which in essence are a bunch of numbers. In technical terms, we'd say that the transformation implemented by a layer is parameterized by its weights (weights are also sometimes called the parameters of a layer). In this context, learning means finding a set of values for the weights of all layers in a network, such that the network will correctly map example inputs to their associated targets. But, a deep neural network can contain tens of millions of parameters. Finding the correct values for all of them may seem like a daunting task, especially given that modifying the value of one parameter will affect the behaviour of all the others.
To control something, first you need to be able to observe it. To control the output of a neural network, you need to be able to measure how far this output is from what you expected. This is the job of the loss function of the network, also called the objective function. The loss function takes the predictions of the network and the true target (what you wanted the network to output) and computes a distance score, capturing how well the network has done. A loss function measures the quality of the network's output.
Input X
|
|
Weights --> Layer (data transformation)
|
Weights --> Layer (Data transformation)
|
|
Predictions Y --> Loss Function <--- True targets Y
|
|
Loss Score
The fundamental trick in deep learning is to use this score as a feedback signal to adjust the value of the weights a little, in a direction that will lower the loss score for the current example. This adjustment is the job of the optimizer, which implements what's called the Backpropagation algorithm: the central algorithm in deep learning.
Initially, the weights of the network are assigned random values, so the network merely implements a series of random transformations. Naturally, its output is far from what it should ideally be, and the loss score is accordingly very high. But with every example the network processes, the weights are adjusted a little in the correct direction, and the loss score decreases. This is the training loop, which, repeated a sufficient number of times (typically tens of iterations over thousands of examples), yields weight values that minimize the loss function. A network with a minimal loss is one for which the outputs are as close as they can be to the targets: a trained network. Again, it's a simple mechanism that, once scaled, ends up looking like magic.
Deep learning has achieved the following breakthroughs, all in historically difficult areas of machine learning:
Deep learning
Deep learning is a specific subfield of machine learning: a new take on learning representations from data that puts an emphasis on learning successive layers of increasingly meaningful representations. How many layers contribute to a model of the data is called the depth of the model (layered representations learning or hierarchical representations learning). Modern deep learning often involves tens or even hundreds of successive layers of representations -- and they're all learned automatically from exposure to training data. Meanwhile, other approaches to machine learning tend to focus on learning only one or two layers of representations of the data; hence, they're sometimes called shallow learning.
In deep learning, these layered representations are learned via models called neural networks, structured in literal layers stacked on top of each other. The term neural network is a reference to neurobiology, but although some of the central concepts in deep learning were developed in part by drawing inspiration from our understanding of the brain, deep-learning models are not models of the brain. There's no evidence that the brain implements anything like the learning machanisms used in modern deep-learning models. For our purposes, deep learning is a mathematical framework for learning representations from data.
The network transforms the digit image into representations that are increasingly different from the original image and increasingly informative about the final result. You can think of a deep network as a multistage information-distillation operation, where information goes through successive filters and comes out increasingly purified )that is, useful with regard to some task). So, deep learning is, technically: a multistage way to learn data representations. It's a simple idea -- but, as it turns out, very simple mechanisms, sufficiently scaled, can end up looking like magic.
Machine learning is about mapping inputs (such as images) to targets (such as the label "cat"), which is done by observing many examples of inputs and targets. Deep neural networks do this input-to-target mapping via a deep sequence of simple data transformations (layers) and that these data transformations are learned by exposure to examples.
The specification of what a layer does to its input data is stored in the layer's weights, which in essence are a bunch of numbers. In technical terms, we'd say that the transformation implemented by a layer is parameterized by its weights (weights are also sometimes called the parameters of a layer). In this context, learning means finding a set of values for the weights of all layers in a network, such that the network will correctly map example inputs to their associated targets. But, a deep neural network can contain tens of millions of parameters. Finding the correct values for all of them may seem like a daunting task, especially given that modifying the value of one parameter will affect the behaviour of all the others.
To control something, first you need to be able to observe it. To control the output of a neural network, you need to be able to measure how far this output is from what you expected. This is the job of the loss function of the network, also called the objective function. The loss function takes the predictions of the network and the true target (what you wanted the network to output) and computes a distance score, capturing how well the network has done. A loss function measures the quality of the network's output.
Input X
|
|
Weights --> Layer (data transformation)
|
Weights --> Layer (Data transformation)
|
|
Predictions Y --> Loss Function <--- True targets Y
|
|
Loss Score
The fundamental trick in deep learning is to use this score as a feedback signal to adjust the value of the weights a little, in a direction that will lower the loss score for the current example. This adjustment is the job of the optimizer, which implements what's called the Backpropagation algorithm: the central algorithm in deep learning.
Initially, the weights of the network are assigned random values, so the network merely implements a series of random transformations. Naturally, its output is far from what it should ideally be, and the loss score is accordingly very high. But with every example the network processes, the weights are adjusted a little in the correct direction, and the loss score decreases. This is the training loop, which, repeated a sufficient number of times (typically tens of iterations over thousands of examples), yields weight values that minimize the loss function. A network with a minimal loss is one for which the outputs are as close as they can be to the targets: a trained network. Again, it's a simple mechanism that, once scaled, ends up looking like magic.
Deep learning has achieved the following breakthroughs, all in historically difficult areas of machine learning:
- near-human-level image classification
- near-human-level speech recognition
- near-human-level handwriting transcription
- improved machine translation
- improved text-to-speech conversion
- digital assistants such as Google Now and Amazon Alexa
- near-human-level autonomous driving
- improved ad targeting, as used by Google, Baidu, and Bing
- improved search results on the web
- ability to answer natural-language questions
- superhuman Go playing
Although deep learning has led to remarkable achievements, expectations for what the field will be able to achieve in the next decade tend to run much higher than what will be possible. Although some world-changing applications like autonomous cars are already within reach, many more are likely to remain elusive for a long time, such as believable dialogue systems, human-level machine translation across arbitrary languages, and human-level natural-language understanding. In particular, talk of human-level general intelligence shoudn't be taken too seriously.
Although we may have unrealistic short-term, expectations for AI, the long-term picture is looking bright. We're only getting started in applying deep learning to many important problems for which it could prove transformative, from medical diagnoses to digital assistants. Your doctor doesn't yet use AI, and neither does your accountant. You don't use AI technologies yet in your day-to-day life, except, you can ask your smartphone simple questions and get fairly useful product recommendations on Amazon.com, and you can search for "birthday" on Google Photos and instantly find those pictures of your daughter's birthday party last month. that's a far cry from where such technologies used to stand. But such tools are still only accessories to our daily lives. AI has yet to transition to being central to the way we work, think and live. Right now, it may seem hard to believe that AI could have a great impact on our world, because it isn't yet widely deployed. But make no mistake: AI is coming in not-so-distant future, AI will be your assistant, even your friend; it will answer your questions, help educate your kids, and watch over your health. It will deliver your groceries to your door and drive you from point A to point B. It will be your interface to an increasingly complex and information-intensive world. And, even more important, AI will help humanity as a whole move forward, by assisting human scientists in a new breakthrough discoveries across all sceintific fields, from genomics to mathematics.
Comments
Post a Comment