Neural networks
From neuron to network: weighted sum, bias, activation function, layers, forward propagation, parameter count, and what a single neuron cannot separate.
- 25 cards
- 5 topics
- Free
The outline, topic by topic
Each topic is weighted the way the exam weighs it.
- Concepts
- Questions
- Sort and link
- The numbers
- Understand
A few cards from the deck
Artificial neuron: what is it?
Show the answer
A computing unit that takes the weighted sum of its inputs, adds a bias, then applies a non-linear function.
Output = f(w₁x₁ + w₂x₂ + … + b). The weights w and the bias b are the parameters that training adjusts.
What does this network return for x1 = 1 and x2 = 0?
```python
def relu(z):
return max(0, z)
x1, x2 = 1, 0
h1 = relu(2 * x1 + x2 - 1)
h2 = relu(-x1 + x2 + 1)
y = relu(3 * h1 - 2 * h2 - 1)
print(y)`2630
Show the answer
2h1 = relu(2 + 0 − 1) = 1; h2 = relu(−1 + 0 + 1) = 0; y = relu(3 − 0 − 1) = 2. Each layer uses the outputs of the previous one; without the biases, one would get 6.
Rosenblatt's … is a … neuron: it outputs 1 if the weighted sum, bias included, exceeds zero, and 0 otherwise.
Show the answer
Rosenblatt's perceptron is a threshold neuron: it outputs 1 if the weighted sum, bias included, exceeds zero, and 0 otherwise.
This 1958 model can only draw a line, hence its inability to compute exclusive OR.
Activation function: what is it?
Show the answer
The function applied to a neuron's weighted sum to produce its output, and which brings non-linearity.
Without it, stacking layers would be pointless: a chain of linear computations is still one linear computation.