Weights, Bias, and the Linear Model

The fundamental equation of linear regression is:

y = weight × x + bias

This is just the equation of a straight line, where weight is the slope and bias is the y-intercept. But understanding exactly what each knob does is the foundation of understanding machine learning, because everything in gradient descent is about finding the right values for these two numbers.

What weight does

Weight controls the angle of the line. Think of placing a physical weight on one end of a seesaw: the heavier the weight, the steeper the tilt.

• A weight of 0 makes the line completely flat (horizontal).
• A positive weight makes the line slope upward as x increases.
• A negative weight makes the line slope downward.
• A larger absolute value makes the slope steeper.

What bias does

Bias controls the vertical position of the line, where it sits when x is zero. It is the y-intercept.

• If weight is zero and you only change bias, the line moves straight up and down.
• Bias tells you "how far away from neutral is my starting point."

The name "bias" is fitting: it biases the output up or down regardless of the input.

Interactive demo: explore weight and bias

Use the sliders below to change the weight and bias and see how the line responds. Notice:

• When weight is 0, the line is flat. Change bias to move it up or down.
• The red dot marks the intercept point (where the line crosses x = 0), which always equals the bias.
• As you increase weight, the line tilts. At weight = 1 and bias = 0, y = x (a 45° diagonal).

Any line, two numbers

Here is the key insight: with just these two numbers, you can represent any straight line. Once you know the weight and bias, you can reproduce the line exactly. And once you have the line, you can use it to make predictions:

prediction = weight × new_input + bias

For Galton's data the weight was approximately 0.66 and the bias 23, giving the model:

predicted_child_height = 0.66 × parent_mid_height + 23

Feed in a parent mid-height of 71 inches and you get a predicted child height of about 69.9 inches, even for a family not in the original dataset.

Now the million dollar question

The interactive demo above is sort of cheating, since you are setting the weight and bias manually. In the real world, you have a dataset of (feature, target) pairs and no idea what the right weight and bias are for its corresponding linear equation.

For example, suppose you recorded how many hours each student in a class studied and the exam score they went on to get:

Each row is one (feature, target) pair: the feature is the hours studied, the target is the score. Plotted, these points do not sit perfectly on a straight line (real data never does), but they clearly trend upward. This is what a dataset actually looks like, and here it is only six rows; a real one might have hundreds, thousands, or even millions of them.

The whole idea of machine learning is this: given a pile of data points like these, find the single weight and bias whose line passes as close as possible to all of them at once. That line becomes your trained model. Feed it a number of study hours it has never seen and it predicts a score.

The million dollar question in machine learning is: how do we find the values for weight and bias that make the best predictions on our data?

There are two broad families of answer. We can solve for the best weight and bias directly with a formula, or we can search for them step by step. Both are worth understanding, because the reason the second one wins is the whole reason this series exists, and that face-off is exactly what the next part is about.

Next: Solve It or Search for It: the two ways to find the best weight and bias, why the one-shot formula runs out of road, and why searching step by step powers all of modern machine learning.