Shannon's Channel Capacity for LLM Prompts: A DSP Approach to Token Cost

An LLM context window has a token limit. That limit is a fixed bandwidth. The output quality depends on what fraction of the token budget carries information that the model needs to answer correctly. Filler words, restated context, polite hedging, and pleasantries are noise. Specific instructions, concrete examples, and constraints are signal. The ratio of signal to noise determines whether you get a useful answer or a generic one.

This is not a metaphor. It is Shannon's channel-capacity theorem applied directly:

C = B × log₂(1 + SNR)

where C is the channel capacity (effective information delivered to the model), B is the bandwidth (your token budget, fixed by the model's context window), and SNR is the signal-to-noise ratio. The only lever you have is SNR. Adding more tokens of noise does not increase capacity — it decreases the marginal information per token while consuming budget.

What the Spectrum Analyzer Shows

The free Prompt SNR Spectrum Analyzer takes any prompt and breaks it into sentences. For each sentence it returns:

• Type: signal (carries new information), noise (filler / pleasantry / hedging), or redundant (restates earlier content)
• Information density: 0.0 to 1.0 score for that segment
• Estimated tokens: how much budget the segment consumes

Plus aggregate statistics:

• Overall SNR: weighted across all segments
• Wasted tokens %: how much of your budget went to noise + redundancy
• Top 3 cuts: specific sentences to delete for the highest information gain per byte
• Recommended compression: how to restructure for higher SNR

The visualization renders each segment as a colored bar — green for signal, red for noise, yellow for redundant — with width proportional to information density. It looks like an oscilloscope trace because that is what it is: a frequency spectrum of prompt density across position.

Why Most Prompts Have Terrible SNR

Prompts are written by humans for AI consumers. Humans default to natural prose: greetings, context-setting, hedging, polite framing. Almost none of this matters to the model. The model does not care that you said "please" or "I would appreciate it if you could." The model does not care about context that was already established three messages ago. The model does not benefit from the reassuring fluency of well-formed English between two specific instructions.

What the model needs:

• The role you want it to play (one sentence)
• The constraints it must obey (numbered list)
• The format it must produce (one example)
• The data it must operate on (delimited block)
• The task to perform (one sentence)

That is the SINC-2 6-band format I use in production. It is engineered to maximize SNR for the LLM channel. A typical "natural" prompt that runs 600 tokens compresses to 180 tokens of SINC-2 with no quality loss — often quality improvement, because the structured format also reduces ambiguity.

Nyquist Sampling Applied to Prompts

Shannon's older sibling, the Nyquist sampling theorem, states that you must sample a signal at twice its highest frequency to reconstruct it faithfully. Sample below Nyquist and you get aliasing — the high-frequency content folds back into the low-frequency band as noise that cannot be removed.

The same applies to prompts. Critical instructions need sufficient repetition (samples) to ensure the model picks them up reliably. Sub-Nyquist sampling — mentioning a constraint once in a long prompt — produces aliasing: the model intermittently misses the constraint, and the missed cases look like random failures. The fix is not to add MORE noise. It is to increase the sample rate of the critical signal: repeat the constraint at the right intervals, in the right format.

From a wiki synthesis I built mapping DSP to context-window engineering: "Token sampling rate in context windows. Sample too rarely → miss signal. Too often → waste budget. Token-Nyquist agent IS this theorem applied to text."

How to Use the Analyzer

Three patterns:

1. Audit a production prompt — paste a prompt you use repeatedly. The Analyzer will tell you what fraction of every call is wasted. Multiply by your monthly volume to see your actual waste in dollars.
2. Compare prompt variants — paste an old prompt, note the SNR. Restructure based on the recommendations, paste again, see the new SNR. Iterate.
3. Diagnose flaky prompts — if a prompt produces inconsistent outputs, the Analyzer often reveals that critical constraints are buried in low-density sections where the model under-samples them.

The Production Version

The free Analyzer scores one prompt at a time. The paid service applies the same DSP discipline to your entire prompt corpus, identifies systemic SNR issues across templates, designs the SINC-2 (or equivalent) format optimized for your specific model and task distribution, and ships a measurable token-cost reduction with documented quality preservation. Typical client outcome: 3-10x cost reduction on the top-volume prompt patterns. The math is the same. The deployment is custom.