Model Weights – What They Actually Are and How They Work • GTech Booster

Whenever someone mentions a “7 billion parameter model” or a “70 billion parameter model,” they are almost always talking about the count of weights inside that model. Weights are the actual substance of a trained AI model, the specific numbers that get learned during training and that determine everything about how the model behaves once it is deployed. This article explains exactly what a model weight is, how weights function mathematically inside a neural network, how they get set during training, why weight count has become the industry’s default way of describing model size, and closes with a fully worked numerical example showing weights in action from raw input to final output.

What a Weight Actually Is

At the most basic level, a weight is a single number that determines how strongly one piece of information influences another as it passes through a neural network. A neural network is built from layers of simple computational units, often called neurons or nodes, and every connection between units carries an associated weight. When information flows from one unit to the next, it gets multiplied by the weight on that specific connection before being passed along, and that multiplication is precisely how the network decides how much importance to give any particular piece of incoming information.

A useful, if simplified, way to think about a weight is as a dial controlling the strength of one specific connection inside the network. A weight close to zero means that particular connection contributes almost nothing to the final result, effectively muting that path of information. A large positive weight means that connection strongly pushes the output in one direction, while a large negative weight means it strongly pushes the output in the opposite direction. A trained model with billions of parameters is really just billions of these dials, each one tuned to a specific value during training, working together across many layers to transform raw input into a useful output.

Weights Versus Parameters Versus Biases

These three terms get used somewhat loosely in casual conversation, so it is worth being precise about how they actually relate to each other.

Parameters is the broadest term, referring to every learnable number inside a model, including both weights and biases together. When a company announces a “70 billion parameter model,” it is referring to the total combined count of every weight and bias across the entire network.

Weights specifically refer to the numbers attached to connections between units, the values that get multiplied against incoming data as described above.

Biases are a related but distinct type of learnable number, one added to a unit’s weighted sum rather than multiplied into it, giving each unit additional flexibility to shift its output up or down independently of its inputs. A unit’s output is generally computed as the weighted sum of its inputs plus its own bias term, before that combined result is passed through an activation function.

In everyday conversation, especially in the context of describing model size, “weights” and “parameters” are frequently used interchangeably, even though biases technically make up a small separate portion of the total parameter count. This is a minor imprecision that rarely causes real confusion, since biases are typically a small fraction of a model’s total parameters compared to the weights.

How Weights Actually Work Inside a Neural Network

To understand how weights function, it helps to walk through what happens at a single computational unit inside a network.

A given unit receives some number of inputs, each of which might be raw data, such as a pixel value or a numeric feature, or might be the output of units in a previous layer. Each of these inputs has its own associated weight. The unit computes a weighted sum: it multiplies each input by its corresponding weight, adds all of those products together, and then adds the unit’s bias term on top. This combined value is then passed through an activation function, a mathematical function that introduces non-linearity into the network, allowing it to model complex, non-linear relationships rather than being limited to simple straight-line combinations of its inputs. The result becomes that unit’s output, which then gets passed forward, multiplied by a new set of weights, into the next layer.

This same basic process, weighted sum plus bias, followed by an activation function, repeats at every single unit across every layer of the network. A large language model might have dozens of layers, each containing thousands or millions of these units, connected through an enormous number of individually weighted connections. The specific values of all of these weights, set during training, are what actually encode everything the model has learned, whether that is grammar, factual knowledge, reasoning patterns, or, in an image recognition model, the visual features that distinguish a cat from a dog.

How Weights Get Set: The Role of Training

Weights do not start out meaningful. Before training begins, they are typically set to small random values, or initialized using specific mathematical schemes designed to help training proceed smoothly rather than getting stuck or becoming numerically unstable early on.

Training is the process of gradually adjusting every weight in the network so that, collectively, they produce useful outputs. This happens through an iterative process generally involving these repeated steps:

Forward pass. A batch of training data is fed through the network using its current weights, producing an output.
Loss calculation. The network’s output is compared against the correct, desired output using a loss function, a mathematical measure of how wrong the current output was.
Backpropagation. Using calculus, specifically the chain rule, the network calculates how much each individual weight contributed to that error, producing a gradient for every single weight in the network.
Weight update. Each weight is nudged slightly in the direction that would have reduced the error, scaled by a small step size called the learning rate.

This cycle repeats, often billions of times across enormous amounts of training data, gradually shifting every weight in the network from its initial near-random value toward a value that contributes usefully to accurate outputs. By the end of training, the specific pattern of weights across the entire network encodes everything the model has effectively learned from its training data, though critically, this knowledge exists as a distributed pattern across potentially billions of individual numbers rather than as anything a human could point to and read directly as an explicit fact or rule.

Why Weight Count Has Become the Standard Way to Describe Model Size

When a company describes a model as having 7 billion, 70 billion, or over a trillion parameters, that number is a rough proxy for the model’s raw capacity to store learned patterns. More weights generally means more capacity to represent complex relationships and a larger volume of learned information, though weight count is far from a perfect measure of actual capability, since architecture design, training data quality, and training technique all matter enormously as well, and two models with very different parameter counts can end up with meaningfully different real-world performance in either direction.

Weight count also directly determines a model’s practical resource requirements. Every weight needs to be stored in memory and needs computation performed against it during every forward pass, so a model with more weights generally requires more memory to load, more computational power to run, and more energy to operate, all else being equal. This is precisely why techniques like quantization, reducing the numeric precision used to store each weight, for example storing weights using 8 bits or 4 bits instead of a full 32-bit or 16-bit floating point number, have become so important for making large models practical to run on smaller hardware, since quantization directly shrinks the memory and computational footprint of the exact same set of learned weight values, at some cost to precision.

Open Weights Versus Closed Weights

Whether a model’s weights are made publicly available is one of the most consequential distinctions in how AI models get distributed today. An open weight model publishes the actual trained weight values for anyone to download and run on their own hardware, while a closed model keeps those weight values private, only exposing the model’s behavior through a hosted API or product. This matters enormously in practice, since possessing the actual weights is what allows someone to run a model entirely offline, modify it through further fine-tuning, or inspect its behavior directly, none of which is possible with a closed model no matter how capable it is, since the underlying weights themselves remain the company’s private property.

A Worked Sample: Weights in Action

To make all of this concrete, here is a small, fully worked numerical example showing weights actually functioning inside a tiny artificial neuron, the same basic mechanism that, scaled up by billions of times, powers today’s large language models.

The setup

Imagine a single artificial neuron with three inputs. This could represent, in a wildly simplified toy example, a unit trying to help decide whether to approve a small loan, based on three numeric input features:

Input 1: Applicant’s income, scaled to a value of 0.8
Input 2: Applicant’s existing debt, scaled to a value of 0.3
Input 3: Applicant’s credit history length, scaled to a value of 0.6

Suppose that, after training, this neuron has learned the following weights and bias:

Weight 1 (on income): 1.5
Weight 2 (on debt): -2.0
Weight 3 (on credit history): 1.0
Bias: -0.5

Notice that the weight on debt is negative, meaning higher debt pushes the neuron’s output downward, which matches real-world intuition: more existing debt should make loan approval less likely. The weight on income is the largest positive value, meaning income has the strongest positive influence on the outcome among these three inputs.

Step 1: The weighted sum

The neuron first multiplies each input by its corresponding weight:

Input 1 x Weight 1 = 0.8 x 1.5  =  1.20
Input 2 x Weight 2 = 0.3 x -2.0 = -0.60
Input 3 x Weight 3 = 0.6 x 1.0  =  0.60

Step 2: Sum the results and add the bias

Weighted sum = 1.20 + (-0.60) + 0.60 = 1.20
Add bias:     1.20 + (-0.5)          = 0.70

Step 3: Apply an activation function

Suppose this neuron uses a common activation function called the sigmoid function, which squashes any input into a value between 0 and 1, useful here for interpreting the result as a probability. The sigmoid function is defined as:

sigmoid(z) = 1 / (1 + e^-z)

Plugging in our result of 0.70:

sigmoid(0.70) = 1 / (1 + e^-0.70)
             = 1 / (1 + 0.4966)
             = 1 / 1.4966
             ~= 0.668

Interpreting the result

The neuron’s final output is approximately 0.668, which, if we are interpreting this as a probability of loan approval, suggests a moderately strong lean toward approval, roughly a 67 percent likelihood of approval, given these particular input values and this particular set of learned weights.

What changes if a weight changes

To see the weights actually doing their job, imagine the debt weight had instead been trained to a much more punishing value of -4.0 instead of -2.0, everything else staying the same:

Input 2 x Weight 2 = 0.3 x -4.0 = -1.20
Weighted sum = 1.20 + (-1.20) + 0.60 = 0.60
Add bias: 0.60 + (-0.5) = 0.10
sigmoid(0.10) = 1 / (1 + e^-0.10)
= 1 / (1 + 0.9048)
= 1 / 1.9048
~= 0.525

With that single weight changed, the same applicant, with the exact same income, debt, and credit history values, now produces an output of roughly 0.525 instead of 0.668, a noticeably weaker lean toward approval. Nothing about the applicant changed. Only one internal weight changed, and that single number meaningfully shifted the neuron’s final decision. This is the entire principle behind how training works, scaled up from one tiny neuron with three weights to a modern language model with billions of them: adjusting the specific numeric values of the weights directly and precisely reshapes what the model outputs for any given input.

Wrapping Up

A model weight is, in the end, nothing more exotic than a single number controlling how strongly one piece of information influences another as it flows through a network, multiplied against an input, summed together with many others, and passed through an activation function at every single unit across every layer. What makes modern AI models so capable is not any individual weight being clever on its own, but the sheer scale and precisely tuned interplay of billions of these simple numbers working together, each one nudged into place through the repeated forward passes, loss calculations, and gradient updates of training. The worked example above uses just three weights and one bias to make a single, simple decision, and while a real large language model scales that same basic mechanism up by many orders of magnitude across a vastly more complex architecture, the underlying principle, numbers multiplying and combining to shape an output, remains fundamentally the same at every scale.