Somewhere in his mind

The Reward Isn't the Objective — Part 2: Closing the Loop

Mon, 21 Sep 2026 02:31:00 +0000

This is part 2. Part 1 covered how I ended up here — two surveys, a suspicious sentence about time-inconsistency, and the moment I finally proved the gap I was chasing was real, not just a hunch. If you haven’t read that one, this will still mostly make sense, but the context helps. Same warning as before: this is long. It’s the messy, chronological version, mistakes included.

Where part 1 left off: I had two exact reference policies for a multi-period risk-adjusted investing problem, and I’d just shown that the reward people actually train reinforcement learning agents on doesn’t land on either of them — it’s a third policy, and comparing them by the field’s usual metric (Sharpe ratio) gets the ranking backwards more than 90% of the time. That was the proof that the thing was real. What I still didn’t have was the why, or any sense of how big a deal it actually is once you put a learning algorithm in the loop instead of an exact solver.

Doing it properly: training an actual agent

At this point I could no longer avoid writing the RL agent, since the whole thing needed one term I hadn’t measured yet: how much error comes from imperfect learning, versus the reward itself.

Wrote a simple policy-gradient agent (REINFORCE), deliberately using a policy class that contains the exact surrogate solution, so any gap I measured was really about optimization and not about the agent being unable to represent the right answer.

First version was broken in a boring but real way: training got better for a while, then got worse the longer I trained it. Took me a minute to realize the exploration-noise term was scaling the gradient by one-over-variance-squared, and as I annealed the exploration down, the gradient variance blew up. Fixed by folding that scaling into the learning rate instead. After that, it converged cleanly.

With a trained agent in hand, I could finally write down the full picture — how much of the gap comes from the philosophical problem (time-inconsistency), how much from the reward shape itself, and how much from imperfect learning:

J(pre-committed) - J(learned agent)
    = [price of time-inconsistency]
    + [cost of the surrogate reward]
    + [learning error]

All three terms, measured, summing up exactly to the total gap — no slack, no fudge.

And here’s the finding I actually like best out of the whole project: which term dominates depends on how risk-averse you are. At low risk aversion, the time-inconsistency price dominates completely and reward design barely matters. Push risk aversion up, and there’s a crossover — somewhere around a risk-aversion parameter of 12 in my test market — past which the reward design term takes over and grows, while the time-inconsistency term shrinks. So the honest answer to “does reward shaping actually matter” is: it depends where you are on that curve, and two papers disagreeing about it could just be standing on opposite sides of it without knowing.

The part where I checked my own work and found I was wrong again

I wanted to sanity-check the whole setup against something simple: the classic “bet a fixed fraction of your wealth to maximize long-run growth” reward (Kelly). I’d written, somewhat confidently, that this reward should trivially match the equilibrium policy, because it’s additive and therefore consistent by construction.

Ran it. It doesn’t match at all. Not even close — different shape entirely. Kelly bets a fraction of wealth; the mean-variance equilibrium bets a constant dollar amount. My reasoning had a hole in it: being internally consistent only means dynamic programming is valid for whatever objective you picked. It says nothing about whether that objective happens to be the one you actually wanted. I’d conflated “this reward is well-behaved” with “this reward is correct,” and those are different claims.

Small thing, but it meant a whole limiting case I’d been planning to use as a clean sanity check needed to be un-written and rewritten more carefully. Kind of embarrassing, kind of the whole point of checking your own work before anyone else has to.

The math actually clicks

Somewhere in the middle of all this, I noticed something I should’ve seen earlier: the surrogate reward, being additive, is itself time-consistent, which means its optimal policy satisfies its own clean backward recursion — no numerical search required, an exact recipe. Once I had that in closed form, I could ask sharper questions than “run it and see.”

The best one: is there ever a penalty setting where the surrogate reward exactly reproduces the true equilibrium policy? Worked through the algebra by hand (with some computer algebra help to check myself), and yes — but only under a specific condition (a risk-free return of exactly 1), and at a very specific penalty:

kappa* = phi * (1 - nu)

not phi, the obvious guess. nu here is basically a squared Sharpe ratio, scaled by how often you rebalance. The correction is not small — in my test market it moved the “right” penalty from 1.0 down to about 0.81.

I dug at why the correction has that particular shape, and it turned out to shrink toward zero the more frequently you rebalance — annual rebalancing gives an 11% correction, weekly gives basically none. That also explains something that had been bugging me: why hadn’t anyone written this down already? Because the classic finance literature on this stuff works in continuous time, where the correction is exactly zero by construction. It only shows up once you’re forced into discrete steps — which is exactly the situation every RL agent is actually in.

I also sanity-checked my from-scratch equilibrium recursion against an existing continuous-time formula from the literature, by discretizing time into more and more steps and watching it converge. It did — cleanly, at the rate you’d expect. That was a good feeling; deriving something yourself and then watching it agree with someone else’s known result from a completely different angle is about as close to “this is probably right” as you get without a formal proof.

Two more things I hadn’t planned on

Two extensions I only got to because I kept asking “okay but does this actually matter in practice.”

First: the differential Sharpe ratio, a fancier reward some papers use, involving running estimates of mean and variance. I expected this to need totally new machinery. It didn’t — with some algebra it turns out to just be a variance-penalized reward again, except the penalty isn’t something you choose, it’s whatever the market happens to produce through those running estimates. Which means a practitioner using this reward thinks they’re setting their own risk preference, but they’re actually inheriting one the market picked for them, and nothing in the setup tells them what it is.

(First attempt at measuring this gave a nonsensical number, because the running estimates hadn’t had time to converge over a short horizon — a good reminder that “the code ran and gave a number” is not the same as “the number means anything.”)

Second, and this one mattered more: everything up to this point assumed returns are independent from one period to the next — a real simplification. So I tested what happens if that’s false: fat-tailed returns, clustered volatility, and actual momentum (returns correlated across time). Fat tails and volatility clustering didn’t change anything meaningful — the ranking of policies held up fine. But momentum broke it completely. The ordering flipped, and the pre-committed policy — previously the best — collapsed hardest, because its whole plan was calibrated to a drift assumption that momentum violates, and it has no way to notice and adjust.

That’s a real limitation, not a footnote, and I said so plainly rather than burying it.

The other thing I checked in that same pass mattered more than I expected: just how invisible is this to a normal backtest? I simulated finite numbers of replications and asked how often a Sharpe-ratio comparison would correctly pick the actually-better policy. At 10 replications: about 2% of the time. At 200 or more: essentially never — and it doesn’t improve with more data, it gets more confidently wrong, because you’re estimating the wrong thing with more and more precision. That’s the sentence I’d point to if someone asked me to summarize the whole project in one line: more backtest data does not fix a misspecified reward. It just makes you more sure of the wrong answer.

Where it stands

I wrote all this up properly — theorems, proofs, an appendix with the derivation, figures, the works. It’s honest about what’s solid and what isn’t. The proof that the surrogate matches the equilibrium exactly at the right penalty: fully proved. The rate at which the mismatch grows away from that special case: proved for the simplest multi-period case, numerically confirmed beyond that. What happens over longer horizons in full generality: still open.

And I still haven’t done a proper, exhaustive search of the finance literature to make sure the central result isn’t already sitting in some paper I haven’t found — I have a good structural argument for why it might genuinely be new (that continuous-time-versus-discrete-time thing), but an argument for why something should be undiscovered isn’t the same as confirming it is.

If you got this far — through both parts — thank you, genuinely, this was long. All the code, the proofs, the write-up, and a running log of every mistake I made along the way (there were more than I expected, and I tried to leave them all visible rather than tidy them away) is here:

github.com/MattTheGreat-root/MVRL-Surrogate-GAP

Everything runs on a laptop in a few minutes, except the full training sweep, which takes about ten. If you find a hole in it, I’d genuinely like to hear about it — several of the best parts of this project came from exactly that.

The Reward Isn't the Objective — Part 1: Finding the Crack

Wed, 16 Sep 2026 02:31:00 +0000

Heads up before you start: this is long, and it’s part one of two. What began as “let me write a research proposal” turned into weeks of proofs, broken code, and one afternoon where I found out something I’d already written down was flat wrong. If you’re just here for the punchline, there’s a part 2 coming with the actual results and the repo link. If you want the whole path, make some coffee.

How this started

I was doing my usual late-night Friday deep dive, casually researching my interests to fill the void in my soul. Once I zeroed in on a specific topic, my friend (an AI, naturally) chimed in: why not make this official and write a research proposal? But “writing a proposal” is a totally different game from “studying AI,” which is my actual strong suit. I’d done the hands-on ML stuff before—scikit-learn, PyTorch, the standard toolkit—but reinforcement learning was brand new to me, and applying it to finance was a completely foreign language.

The topic was eventually: reinforcement learning for asset allocation. So I did what you’re supposed to do and started reading surveys.

The first one was Bai et al., a 2024 review of RL in finance. It’s the kind of paper that’s genuinely useful when you’re starting from nothing — it catalogues dozens of papers, groups them by application (market making, portfolio management, optimal execution), and does a meta-analysis of what actually seems to work. I read it end to end, mostly to build vocabulary.

Buried in there was a table. It classified the rewards people use to train these agents. Plain return. Profit and loss. Sharpe ratio. Differential Sharpe ratio. Variance penalties. Normal stuff, nothing alarming on the surface.

Then there was a sentence that stuck with me more than it probably should have: the survey compared studies using plain return against studies using a “shaped” (risk-adjusted) reward, and said the shaped ones perform better. The p-value for that comparison was 0.120.

That’s not significant. And it’s not even a fair comparison — it’s pooled across papers using different markets, different periods, different algorithms, different everything. I remember thinking: so nobody has actually tested this cleanly. Everyone just… does it, and reports a backtest number.

I didn’t think much more of it at the time. I filed it away and moved on to writing a normal proposal — pick a reward, sweep some hyperparameters, run PPO, report Sharpe ratios. Standard stuff. I even got pretty far into planning the experiment before something else got in the way.

The second survey, and the sentence that changed the whole plan

Someone told me to look at Hambly, Xu, and Yang’s 2023 survey too — the more mathematical one, published in Mathematical Finance rather than an ML venue. Different audience, different rigor.

And there it was, almost in passing: the mean-variance framework — the thing basically everyone means when they say “risk-adjusted” in finance — is time-inconsistent. A strategy that’s optimal today is not optimal tomorrow, even if nothing has changed except that time has passed. And because of that, the Bellman equation doesn’t hold for it.

I had to sit with that for a minute. Every reinforcement learning algorithm I knew — Q-learning, actor-critic, PPO, all of it — is built on the assumption that you can solve a long problem by solving the next step correctly and then continuing optimally. That’s the whole trick. That’s what makes RL work. And here was a paper telling me, almost as an aside, that the objective everyone in the first survey was implicitly chasing doesn’t support that trick at all.

So when someone trains PPO on a Sharpe-ratio reward, sums it over time, and calls the result “risk-adjusted” — what have they actually built? Not the thing they think they built. The algorithm is correct, in the sense that it correctly optimizes the sum it was given. It’s the sum itself that isn’t the thing.

That’s when the proposal I’d half-written stopped feeling interesting. Sweeping hyperparameters on a reward nobody had checked the meaning of felt like measuring something without knowing what the ruler was calibrated to.

First proposal, first rejection (sort of)

I wrote a proposal anyway — a controlled comparison of reward functions, decent statistics, robustness checks. I thought it was solid. It got pushback: no clear method, the gap wasn’t obviously real, and — fair — not much actual math in it. Just prose about what I’d measure.

That stung a little, but it was the right call. I’d been describing an experiment and calling it a contribution. Those aren’t the same thing.

Finding the actual math

I went back to the setting properly this time. Multi-period mean-variance, the Li & Ng (2000) formulation — you get a pre-committed optimal policy in closed form, solved by embedding the problem in an auxiliary one. It’s affine in wealth: u*(x) = alpha*x + beta. Clean.

But because of the time-inconsistency thing, there’s a second notion of optimal: the equilibrium policy — the one where no future version of you wants to deviate from the plan, given everyone else sticks to it too. Basak and Chabakauri worked this one out. It’s a genuinely different policy from the pre-committed one, not just a variant.

So now I had two reference points. Two different “correct” answers to the same question, depending on what you mean by correct.

And then the actual question crystallized: when you train RL on a summed-up risk-adjusted reward, which of these — if either — do you land on?

Nobody had answered that. Not because it’s obscure, but because you need both closed forms to check against, and most work just runs a backtest and reports whatever number comes out.

Building it, and immediately breaking it

I started coding. First the pre-committed policy — the Li-Ng closed form. Then a verification step, because I didn’t trust myself: I threw a completely independent numerical optimizer at the same problem, with zero knowledge of the closed form, from random starting points. If they agreed, I’d believe it.

They didn’t, at first. My first check used Nelder-Mead in 24 dimensions, and it just… didn’t converge. It failed silently in a way that made the test look like it passed, because all I was checking was “did the optimizer beat my closed form,” and a non-convergent optimizer obviously can’t beat anything. That’s a test that passes even when it’s meaningless. I only caught it by looking too closely at the numbers and going “wait, that doesn’t look optimized at all.”

Swapped in L-BFGS-B with multiple restarts, required agreement in both directions. Then it worked — agreement to about 2.6e-12. That felt good. First real “okay, I trust this” moment.

Then I built the equilibrium policy — a genuine backward recursion, trickier than the pre-committed case. Checked it against its own definition directly: at a bunch of wealth levels and time steps, I asked an optimizer to find a better single-period deviation than what my policy prescribed, given everyone else keeps playing the equilibrium strategy. Best improvement found: zero, to machine precision.

There was a nice surprise buried in there too. The equilibrium policy turned out to hold a constant dollar amount in risky assets — not a constant fraction of wealth, an actual constant number — while the pre-committed one scales down as wealth grows. Different shapes, not just different numbers. That’s a known result in the literature, but reproducing it from scratch was itself reassuring: if my recursion had a bug, it almost certainly wouldn’t have reproduced a known structural fact by accident.

The part where I almost shipped something false

Somewhere in here I got curious about why time-inconsistency happens, mechanically, not just abstractly. So I ran an experiment: compute the optimal plan at time zero, let one period pass, then re-solve the exact same problem from scratch at time one. Compare what the original plan says to do against what the fresh solve says to do.

They disagreed — expected, that’s the whole point. But the disagreement had a very specific shape: it was a straight line in wealth, and it crossed exactly zero at one particular wealth level. I wrote down, casually, that the disagreement was “present everywhere, especially in the tails.” Then I actually plotted it and found the opposite: it vanishes exactly at the wealth level your past self expected you’d be at, and grows the further you are from that.

That’s a much better fact than the one I almost wrote down. The mechanism turned out to be really clean too — the whole plan is anchored to a multiplier that’s pinned by your expected terminal wealth, so if reality tracks the plan exactly, nothing needs to change. It’s only surprise that breaks consistency. I liked that a lot more than the vague thing I’d have shipped if I hadn’t double-checked myself.

Does the actual gap even exist

At this point I had two exact reference policies but still hadn’t shown the thing my whole idea depended on: that training RL on a summed reward gets you somewhere else.

Here’s the trick that made it possible to check this without touching reinforcement learning at all: the reward practitioners actually write down — “return minus a penalty on the square of return” — turns out to also have a closed-form optimum, because summed-up rewards are automatically time-consistent (sums always are). So I could compute the exact maximizer of the practitioner’s reward and just… compare it directly to the two references. No training, no noise, no excuses.

First thing I checked: the plain-return reward, no risk penalty at all — the single most common choice in the surveyed literature. It has no maximum. At all. Scale up your bets and the expected reward just climbs forever, 0.88, 8, 80, 800, 8000, 80000. The actual risk-averse problem is perfectly well-behaved and bounded; the reward that’s supposed to represent it isn’t. Anyone training on it isn’t solving a hard version of the right problem. They’re solving a problem with no answer, and what they get back is whatever their leverage limits happened to allow.

With an actual penalty added in, there is a finite optimum — and it’s neither of my two reference policies. It’s a third thing. And the gap wasn’t even one-directional: at the best possible penalty, this “surrogate” policy actually slightly beat the equilibrium one on its own terms. Not a worse version of the equilibrium policy — a genuinely different object.

And then the part that made me want to write this whole post: I compared these policies by Sharpe ratio, the thing everyone reports in backtests. The naive choice — set your reward’s risk penalty equal to your actual risk aversion, which is the obviously reasonable thing to do — had a higher Sharpe ratio than the true equilibrium policy, while having a lower value on the actual objective. Backwards. If you evaluated these by the field’s normal method, you’d conclude the wrong one is better, on precisely the dimension where it’s worse.

I checked that wasn’t a fluke by running it across about 500 random markets and risk-aversion levels. The reversal shows up over 90% of the time.

That’s where part 1 ends — the point where I finally had proof, not just a hunch, that the thing I’d suspected two surveys ago was real. Part 2 picks up with actually training an agent, a decomposition that splits the error into three honest pieces, a mistake I nearly shipped involving a textbook betting strategy, and a result about backtests that I think is the sharpest thing to come out of the whole project. Repo link is at the bottom of part 2.

Invictus

Sun, 30 Aug 2026 02:31:00 +0000

Out of the night that covers me,
Black as the pit from pole to pole,
I thank whatever gods may be
For my unconquerable soul.

In the fell clutch of circumstance
I have not winced nor cried aloud.
Under the bludgeonings of chance
My head is bloody, but unbowed.

Beyond this place of wrath and tears
Looms but the Horror of the shade,
And yet the menace of the years
Finds and shall find me unafraid.

It matters not how strait the gate,
How charged with punishments the scroll,
I am the master of my fate,
I am the captain of my soul.

Project Purple

Wed, 19 Aug 2026 14:17:00 +0000

In 2005, the smartphone industry was completely obsessed with plastic. If you look back at a BlackBerry or a Palm Treo from that era, the screen was almost an afterthought. The entire bottom half of the device was dominated by a fixed, cramped grid of little plastic chiclets. To the rest of the industry, this was just how “serious” business phones were built. To Steve Jobs, it was a massive design flaw.

Jobs hated the idea of static hardware dictating the software. His vision for the iPhonecodenamed “Project Purple”was ruthless in its simplicity. Why should a phone carry around a plastic keyboard when you’re trying to watch a video? In that context, those buttons are just dead space.

Jobs wanted a device that was nothing but a screena blank slate where the interface could instantly morph to fit whatever app you were using. You get a calculator pad when you need numbers, a full QWERTY layout when you need to type, and absolutely nothing when you want to watch a movie. The hardware had to get out of the way.

It was a brilliant product vision. But there was just one massive problem: typing on a flat sheet of glass was terrible.

So many things wrong with glass

Early prototypes of the iPhone were failing at the most basic hurdle. Without the tactile “click” of a physical button, users were totally lost. Human thumbs are simply too wide and blunt to accurately hit tiny, 40-pixel targets on a smooth surface. Inside Apple, testers were trying to tap out simple emails and ending up with strings of pure gibberish. The error rates were off the charts.

For Jobs, giving in and bolting a plastic keyboard onto the bottom of the iPhone wasn’t an option. It would ruin the purity of the device. But if his team couldn’t figure out how to make typing on glass actually workif they couldn’t somehow make it accurate and intuitivethe whole concept of a screen-only smartphone was completely dead in the water.

The “Keyboard Derby”

Recognizing the existential threat to the project, Apple software executive Scott Forstall hit the panic button. He paused nearly all other user interface work on the iPhone and initiated a brutal internal competition.

For three weeks, every software engineer on the UI team was tasked with doing just one thing: build a touchscreen keyboard that actually worked.

The engineers pitched wildly different concepts. Some threw out the standard QWERTY layout entirely, arguing it wasn’t made for thumbs anyway. Others tried grouping multiple letters onto single, larger keys to compensate for inaccuracy.

But the winning design came from a principal engineer named Ken Kocienda. Kocienda realized a fundamental truth of product design: you don’t force users to learn a completely new layout while simultaneously forcing them to learn a completely new input method (glass).

The visual layout had to remain standard QWERTY. The innovation couldn’t be in the visible design. It had to be invisible.

Kocienda’s breakthrough was recognizing that the keys didn’t need to change visuallythey needed to change behaviorally in the background, outsmarting the user’s fat fingers in real-time.

The Phantom Hitboxes: An Algorithm in Disguise

To understand the genius of the original iPhone keyboard, you have to separate the UI from the underlying architecture. What the user saw was a static, unmoving grid of QWERTY keys. But what the iPhone’s processor saw was a highly dynamic, probabilistic data pipeline that was constantly shifting in real-time.

When you tapped a piece of glass, you weren’t actually pressing a buttonyou were simply feeding an (x, y) coordinate on a 2D plane into an algorithm. Ken Kocienda’s job was to figure out what you actually meant to type based on those coordinates, even when they were objectively wrong.

He solved this by building a localized predictive model that relied on three distinct layers of logic:

1. The Probabilistic Dictionary

At its core, the keyboard functioned as a character-level $n$-gram model, constantly calculating the conditional probability of the next character in a sequence. Rather than treating each keystroke as an isolated independent event, the software relied on a Markov assumption—specifically that the probability of the next character $c_i$ depends on the preceding $n-1$ characters.

Formally, the probability distribution over the vocabulary for the next character is evaluated as:

\[P(c_i \vert{} c_1, c_2, \dots, c_{i-1}) \approx P(c_i \vert{} c_{i-(n-1)}, \dots, c_{i-1})\]

In practice, if you tapped the letter “T” ($c_1$) and then “H” ($c_2$), the system instantly queried its local lexicon to calculate the Maximum Likelihood Estimation (MLE) for every possible subsequent character. Using a trigram context, the probability of the next letter being “E” is determined by the historical frequency counts within the English language corpus:

\[P(c_3 = \text{'E'} \mid c_1 = \text{'T'}, c_2 = \text{'H'}) = \frac{\text{Count}(\text{'T'}, \text{'H'}, \text{'E'})}{\text{Count}(\text{'T'}, \text{'H'})}\]

Because the sequence “THE” is statistically dominant, this equation yields an extremely high probability scalar. Conversely, evaluating the likelihood of an impossible sequence—such as $P(c_3 = \text{‘Q’} \mid \text{‘T’}, \text{‘H’})$—evaluates to near zero. These calculated probability weights were then used as the direct mathematical inputs to reshape the actual touch zones on the screen.

2. The Dynamic Hit Targets

This is where the magic happened. Based on those calculated probabilities, the software instantly redrew the invisible touch-sensitive regions (the hitboxes) for every key on the screen.

High-Probability Keys: The hitboxes for “E”, “A”, and “I” would massively expand, silently cannibalizing the pixels of the surrounding keys.
Low-Probability Keys: The hitboxes for keys like “Q” and “Z” would shrink down to a fraction of their visual size.

If you were trying to type “THE”, but you were moving fast and your clumsy thumb physically landed on the left edge of the “R” key, the phone wouldn’t register an “R”. Because the “E” hitbox had expanded to overlap the physical drawing of the “R” key, the algorithm confidently registered an “E”.

3. Geometric Distance & “Keyboard Constellations”

Simply resizing hitboxes wasn’t enough to handle truly terrible typing. To fix mangled words, Kocienda engineered a geometric pattern-matching system.

Every time you typed a word, your sequence of taps created a specific path of (x, y) coordinates across the screenwhat Kocienda called a “keyboard constellation.” If you typed a word so poorly that the dynamic hitboxes couldn’t save it, the system would look at your messy constellation of coordinates and calculate the Euclidean distance between your raw taps and the “ideal” coordinates for actual dictionary words.

The software would generate a list of candidate words that matched your geometric shape, and then apply Zipf’s Law (a statistical rule about how frequently certain words are used in a language) to weight the results. The most statistically probable word with the closest geometric distance won the tiebreaker.

The user was completely blind to this entire pipeline. To them, it just felt like the glass keyboard was somehow reading their mind.

Trusting the Math

All of this invisible maththe probabilistic dictionaries, the shifting hitboxes, the geometric constellation mappingwas an engineering triumph. But it created a massive psychological problem.

How do you get a user to trust a piece of glass that is secretly changing the rules of the keyboard while they type?

The answer became the most famous (and sometimes infamous) user interface element of the modern smartphone era: the autocorrect suggestion bubble.

Breaking the “Context Switch”

During internal testing, Ken Kocienda noticed a fatal flaw in how people interacted with the touchscreen. Because the glass lacked tactile feedback, users didn’t trust it. They would peck a letter, look up at the text field to make sure the right letter appeared, and then look back down at the keyboard.

If they made a mistakesay, hitting the “R” key instead of the “E”they immediately hit the backspace key to fix it.

Kocienda called this the “context switch,” and it absolutely broke his algorithm. The geometric constellation mapping needed the user to finish typing the entire word. If you only typed two letters, realized you messed up, and deleted one, the software didn’t have enough geometric data points to guess what word you were actually aiming for.

Apple realized they didn’t just have to design new software; they had to reprogram human behavior.

The autocorrect UI was designed to encourage momentum. As you typed your mangled, fat-fingered gibberish, the algorithm’s best guess would hover in a little bubble just below your text. The genius was in the execution: you didn’t have to tap the bubble to accept the correction. You just hit the spacebar.

Apple’s marketing and instructional videos for the original iPhone all hammered home the same core message: Just type. Don’t look down. Don’t stop to fix mistakes. Trust the keyboard.

If you just kept your thumbs moving, the algorithm would gather enough coordinate data to map your constellation, apply Zipf’s Law, and seamlessly swap your gibberish for the correct word the moment you hit the spacebar.

The BlackBerry Killer

When Steve Jobs walked onto the Macworld stage in January 2007, the executives at companies like Research In Motion (the makers of BlackBerry) were watching closely. Many of them believed typing on glass was a physics problem that couldn’t be solved. They assumed the iPhone would be a neat toy, but useless for serious enterprise email.

Then Jobs demonstrated the keyboard. He didn’t use a stylus. He used his bare fingers, typing quickly and fluidly. The autocorrect engine caught his errors silently and perfectly.

In that single presentation, the fixed physical keyboard became obsolete.

The End of Plastic

It’s easy to laugh at autocorrect today, usually when it aggressively changes “fucking” to “ducking” in a text message. But in 2007, that invisible safety net wasn’t a jokethat was the only thing keeping the iPhone from being a very expensive paperweight.

What Kocienda really figured out during that Keyboard Derby was that typing on a touchscreen isn’t actually typing at all. There are no buttons. There is no physical input. You are simply throwing raw spatial data at a statistical engine and trusting the math to figure out what you meant.

By letting an algorithm do the heavy lifting that plastic keys used to do, Apple proved that good software could completely mask the physical flaws of a piece of hardware. It’s the reason the physical smartphone keyboard died almost overnight, and it’s the reason you aren’t carrying a BlackBerry right now.

Stalker: A Public Scoreboard for People

Fri, 31 Jul 2026 14:17:00 +0000

You just met someone.

Could be a girl you’re thinking about dating, a guy you’re about to do a group project with, or some new coworker who already talks too much. They seem fine. Maybe even good.

But you have no idea if they’re actually full of shit.

You don’t know if the last three people who dated them walked away thinking “never again.” You don’t know if their previous teammates quietly prayed they’d drop the class. You don’t know if they’re the type who shows up or the type who ghosts the second work gets real.

That information exists. It’s just locked in group chats, private rants, and the faces of people who already got burned.

Stalker is the public version of that information. → Try it here

A big, ugly database of numbers slapped onto real people — looks, personality, style, intelligence, money, body. Crowdsourced scores from the people who actually know them. No comments. No essays. Just the numbers.

It’s the digital equivalent of leaning over to the person next to you and asking:

“Be honest. Is this one worth it or should I run?”

Some people will call it toxic.
Some will get their feelings hurt when they see their score.

I don’t really care.
I built it anyway.

The Concept: A Shadow Transcript

Your official transcript says someone got an A in Data Structures.
It doesn’t say whether they carried the group project, ghosted until 3 a.m. the night before, or submitted code that looked like it was written by a raccoon.

Stalker fills that gap. Crowdsourced reputation. A living, breathing ranking of the people you actually have to work with.

Is it fair? Probably not.
Is it interesting? Extremely.

The Math: Seed Scores and the “Clamp Lag” Problem

Raw averages are trash for this kind of system. New profiles get destroyed by the first three angry votes, and the whole thing becomes a popularity contest that swings wildly.

So every profile starts with a seed score (I went with 8).
Users then vote +1 or –1 on each metric.
The backend just does:

Seed + total upvotes – total downvotes

Simple. Honest. Trackable.

But the UI has a hard clamp: scores never show above 10 or below 0.

Here’s where it gets spicy.

Imagine someone has a seed of 8 and has already collected five upvotes. True score = 13. The UI shows a clean 10.
Someone comes along and downvotes them. True score drops to 12.
UI still shows 10.

To the person who just voted, the number didn’t move. It feels broken. That’s Clamp Lag — the price you pay for bounding a cumulative sum while still tracking every individual vote.

You could switch to a shared 0–10 slider that updates instantly, but then you lose the ability to know who voted and how the score actually evolved. In a system built on judgment, accountability matters more than instant gratification.

The Stack and the Production Nightmares

Django backend. Vanilla JavaScript frontend. SQLite database. Hosted on a persistent PythonAnywhere server.

This is where the real education happened.

Lesson 1: The Git Database Wipe

Early on I committed db.sqlite3 like an absolute rookie.
One git pull --hard later and every user, every rating, every profile vanished. Git happily overwrote the live database with the empty one from the repo.

Fix:

db.sqlite3 goes in .gitignore
git rm --cached db.sqlite3
Daily physical backups of both the database and the media folder

GitHub is for code. It is not your backup strategy.

Lesson 2: Ghost Sessions

After the wipe, the frontend still had old user IDs sitting in Local Storage.
It would successfully create a new profile, then try to attach ratings using a user that no longer existed. Django returned 400s. The JavaScript ignored them and redirected anyway. Result: profiles stuck at zero with no way to recover.

Now the frontend actually checks the response before it does anything. Revolutionary, I know.

Lesson 3: The Golden Backup Ritual

Any migration that touches the schema is high-stakes.
My deployment ritual is now religious:

Branch the changes
SSH in and run cp db.sqlite3 db_safety_snapshot.sqlite3
Pull the new code
Migrate and collectstatic

If it explodes, the abort button is instant: delete the broken DB, rename the snapshot, checkout main. Sleep well.

At the end of the day, this whole thing is just numbers on people.

Some will say it’s toxic. Some will say it’s unfair. Some will get their feelings hurt when they see a 4.2 next to their name and realize the room doesn’t think as highly of them as they do.

Good.

The world already judges you. It just does it quietly, behind your back, in group chats you’ll never see. Stalker just makes the judgment visible and permanent.

Is that dystopian? Probably.
Is it useful? Sometimes.
Is it going to make some people uncomfortable? Definitely.

I didn’t build this to make friends or fix society.
I built it because I wanted to see what happens when you give students a public scoreboard for each other and let the numbers do the talking.

The database is live.
The scores are moving.
And if yours is low, well… now you know.

→ Try Stalker here

Dev-Log: Breaking the Rubika Web SPA (A Descent into Madness)

Mon, 06 Jul 2026 03:20:00 +0000

03:20 AM, deep flow state.

Espresso number three is keeping the lights on. Selenium has trust issues, Angular has declared war on me, and I’m becoming increasingly convinced the DOM tree knows exactly what it’s doing.

Let’s build a headless Rubika broadcaster.

First, we thought it would be easy. Boot up Selenium, log in, search for the user, click the “Message” button, and hit send. It’s just basic web automation, right? Find element by XPath, send keys, click enter. Five minutes, in and out.

Oh, how naive. I am documenting this descent into madness so I never forget the absolute chaos of trying to automate a modern, aggressively sandboxed Single Page Application (SPA). Here is the exact anatomy of how this app fought back at every single step, and the increasingly unhinged lengths we had to go to in order to break it. You can see in the text that I keep using the word “We” which is refereeing my team (me, Claude and gemini).

Phase 1: The Frankenstein SPA & The Offline Mode Curse

Our first goal was persisting the login session. Nobody wants to scan a QR code or type an SMS OTP every time they run a script. I tried the classic route: manually ripping out the authentication cookies and injecting LocalStorage into the driver using a saved JSON file.

The Problem

The bot would launch, bypass the login screen perfectly, and immediately crash into a terrifying “half-loaded” state. The green background rendered, but the UI was shattered. The console was screaming about missing assets. Why? Because Web 3.0 SPAs don’t just use cookies anymore. They rely heavily on IndexedDB for chat histories and service workers for state routing. By injecting LocalStorage without the accompanying local databases, we created a Frankenstein browser state. The framework woke up, realized half its brain was missing, assumed we had no internet connection, and threw itself into a fatal offline loop.

The Fix

We threw away the manual injection entirely and stopped trying to outsmart the browser. We switched to a Native Chrome Profile (--user-data-dir). We literally just told Chrome, “Hey, save your own databases to this folder.” The next time the bot spun up, it was natively logged in with perfectly synced, uncorrupted databases. The UI loaded flawlessly. One crisis averted.

Phase 2: The Virtualized Scrolling Nightmare (The Disappearing DOM)

We survived the login. Next up: building the Data Scraper to extract post prices and engagement metrics from the shop’s profile grid. We just needed to scroll down the user’s feed, click each post, grab the data, and go back. A standard body.scrollHeight loop, right? Absolutely not.

The Problem

Rubika uses a highly aggressive virtualized grid to save memory. As you scroll down, the app actively deletes older posts from the raw HTML. If we told Selenium to scroll to the bottom, the posts vanished from RAM before the bot could even click them. If we tried to iterate through a pre-counted list of 50 posts, the script crashed with a StaleElementReferenceException by post #12 because the target no longer existed.

To make matters worse, the UI renders the grid twice during CSS transitions. This leaves invisible “ghost” tiles sitting in the code with stale X/Y coordinates. When Selenium blindly clicked the first <img> tag it found, it clicked the ghost, triggering a fatal ElementClickInterceptedException.

The Fix

After burning far too many hours trying every conventional Selenium trick in the book, I handed the problem to Claude. It spent quite a while iterating through different approaches before finally landing on a solution that actually understood how Rubika’s virtualized grid behaved.

We had to completely abandon traditional scraping and build a custom spatial-awareness engine.

First, we stopped scrolling globally. We wrote a custom JavaScript injector to target the exact hidden scrollable container (.rtl-l3f8vc) and scroll by exactly 250 pixels at a time—just enough to load the next row without un-mounting the current one.

Second, we stopped trusting the HTML. We used JavaScript getBoundingClientRect() to map out the physical X/Y coordinates of every tile on the screen, deduplicating them by their actual location on the monitor rather than their DOM tags. Finally, we used document.elementFromPoint(cx, cy) to shoot a virtual raycast at the center of the tile. If the ray hit an invisible overlay instead of the post, the script knew it was a ghost, ignored it, and moved on.

We beat the grid.

Phase 3: Angular’s State Manager (The Invisible Wall)

Okay, so we are logged in. We successfully navigated to a profile via the Vitrin search and used a hardware-level click to trigger the Message button. The chat UI slides open. We told Selenium to locate the text box and drop the payload using .send_keys().

Nothing happened. The box stayed empty. The send button stayed greyed out.

The Problem

Inspecting the DOM revealed an absolute nightmare of frontend engineering.

<textarea name="draftMessage" ng-trim="false" class="ng-untouched ng-pristine" style="display: none;">

Rubika’s web client is heavily driven by Angular. The chat input wasn’t a standard HTML <input> tag; it was a layered <div> with contenteditable="true", sitting under an invisible placeholder span, sitting next to a hidden textarea.

When Selenium forces text into a div via send_keys(), it bypasses the operating system and violently injects the text straight into the DOM. Angular’s state manager (the thing tracking the ng-pristine class) completely ignores this because it demands a literal OS-level hardware keystroke to register a change. We tried ActionChains. We tried native clipboard pasting (pyperclip). The framework swallowed all of it.

The Fix

A pure JavaScript bypass. We had to stop asking Selenium to act like a user and instead hijack the browser’s JavaScript engine directly. We injected this exact payload:

var box = arguments[0];
box.innerText = arguments[1]; // Inject the raw string directly into the node
box.dispatchEvent(new Event('input', { bubbles: true })); // Force Angular to wake up

That dispatchEvent was the silver bullet. We sent a fake, bubbling input event up the DOM tree. Angular intercepted it, updated its internal tracking, flipped the class to ng-dirty, and finally turned the Send button blue.

Phase 4: Chasing Ghost Elements

So the text injection works. Great. We run the script again. It opens the chat, executes the JavaScript, and… the text doesn’t appear. No errors. No crashes. Just silence.

The Problem

SPAs are incredibly lazy. When you navigate away from a chat page, the framework doesn’t delete the old HTML from your RAM; it just hides it by setting display: none;. Selenium was blindly searching the document for //div[contains(@class, 'composer_rich_textarea')]. It was finding an inactive, hidden chat box from a completely different screen that was still buried in the DOM, and silently typing our message into the void while the real, visible chat box sat entirely empty.

The Fix

We built a DOM filter. Instead of targeting one element, we scraped every single text box in the raw HTML and ran a hard .is_displayed() boolean check on each one. The script iterated through the ghosts until it locked onto the active UI element physically visible on the monitor.

Phase 5: The Iframe Sandbox (The Final Boss)

We are in the home stretch. Mode 1 (Targeted User) works perfectly. I moved on to Mode 2 (Broadcast to All Contacts). The bot opens the contact list, scrapes the names, searches them up, clicks their profile, the chat opens… and Selenium throws a fatal 15-second TimeoutException looking for the text box.

I was losing my mind. I could physically see the text box on my screen. Why was Selenium suddenly blind?

I wrote a massive diagnostic block to pause the script, dump the raw HTML to a text file, and take a screenshot of what Selenium’s engine was seeing. The screenshot showed the chat UI perfectly. But when I searched the 1800-line HTML dump for the text box class, I got 0 results.

The Problem

I scrolled to the very bottom of the raw HTML dump and found this monstrosity:

<iframe title="test" src="/messenger/index.html?cache=2.5.8..." sandbox="allow-same-origin allow-scripts..."></iframe>

They put the entire chat interface inside an isolated, sandboxed <iframe>. The profile page is the parent document, but the actual messaging engine renders inside a portal. Selenium was scanning the main parent document, staring blankly at the profile page, completely unaware that the text box was inside the sandbox.

The Fix

We had to breach the sandbox. Before looking for the text box, we commanded Selenium’s internal context to step inside the iframe:

iframe_xpath = "//iframe[contains(@src, 'openchat')]"
self.wait.until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, iframe_xpath)))

Once inside, the entire chat DOM became visible. We dropped the Angular payload, hit ENTER, and then—crucially—executed self.driver.switch_to.default_content() to step back out into the main document so the script wouldn’t crash when it tried to search for the next user.

By the time it was over, the bot could launch a persistent Chrome profile, navigate the SPA, bypass Angular’s input validation, distinguish visible elements from hidden ghosts, step into sandboxed iframes, deliver the message, and continue to the next user without losing its place.

Check out the full project here

None of these problems were difficult because the code was complicated. They were difficult because every layer of the application assumed a human would always be sitting behind a browser. Every step was less about writing automation and more about understanding how the application actually behaved beneath the UI. Goodnight!

Do not go gentle into that good night

Fri, 19 Jun 2026 02:31:00 +0000

Do not go gentle into that good night,
Old age should burn and rave at close of day;
Rage, rage against the dying of the light.

Though wise men at their end know dark is right,
Because their words had forked no lightning they
Do not go gentle into that good night.

Good men, the last wave by, crying how bright
Their frail deeds might have danced in a green bay,
Rage, rage against the dying of the light.

Wild men who caught and sang the sun in flight,
And learn, too late, they grieved it on its way,
Do not go gentle into that good night.

Grave men, near death, who see with blinding sight
Blind eyes could blaze like meteors and be gay,
Rage, rage against the dying of the light.

And you, my father, there on the sad height,
Curse, bless, me now with your fierce tears, I pray.
Do not go gentle into that good night.
Rage, rage against the dying of the light.

I wish there was a war.

Mon, 15 Jun 2026 12:00:00 +0000

Dear Edward,

This just serves to acknowledge receipt of yours per Cap Lowndes which was delivered me yesterday. The truth of Cap Lightbowen & Lowndes information is now verifyd by the presence of your Father and Sister for whose safe arrival I Pray, and that they may convey that satisfaction to your soul that must naturally flow from the sight of absent Friends in health, and shall for news this way refer you to them.

As to what you say respecting your having soon the happiness of seeing us all, I wish, for an accomplishment of your hopes provided they are Concomitant with your welfare, otherwise not, tho doubt whether I shall be Present or not.

For to confess my weakness, Ned, my Ambition is prevalent. That I contemn the grov’ling condition of a Clerk or the like, to which my Fortune &c condemns me and would willingly risk my life though not my Character to exalt my station. I’m confident, Ned that my Youth excludes me from any hopes of immediate Perferment, nor do I desire it, but I mean to prepare the way for futurity.

I’m no Philosopher you see and may be justly said to Build Castles in the Air. My Folly makes me ashamed and beg youll conceal it, yet Neddy we have seen such schemes successful when the Projector is Constant.

I shall Conclude saying I wish there was a war.

I am,
Dr Edward,
Yours,

Alex Hamilton

The Oldest Fear: K2-18b, The Great Filter, and the Silence of the Stars

Sat, 19 Apr 2025 23:00:00 +0000

It was yesterday morning when I first heard the news that the James Webb telescope had found signs of life on a planet outside our solar system. Ngl, I was fascinated. Here it goes:

“K2-18b, located 124 light-years from Earth, could be a Hycean world: a potentially habitable planet entirely covered in liquid water with a hydrogen-rich atmosphere,” said lead study author Nikku Madhusudhan, professor of astrophysics and exoplanetary science at the University of Cambridge’s Institute of Astronomy. “Earlier theoretical work had predicted that high levels of sulfur-based gases like DMS and DMDS are possible on Hycean worlds. And now we’ve observed it, in line with what was predicted. Given everything we know about this planet, a Hycean world with an ocean that is teeming with life is the scenario that best fits the data we have.” — CNN

“The team sidestepped this challenge by analyzing light from K2-18 b’s parent star as it passed through the exoplanet’s atmosphere. K2-18 b is a transiting exoplanet, meaning that we can detect a drop in brightness as it passes across the face of its host star. This is how the exoplanet was first discovered in 2015 with NASA’s K2 mission. This means that during transits a tiny fraction of starlight will pass through the exoplanet’s atmosphere before reaching telescopes like Webb. The starlight’s passage through the exoplanet atmosphere leaves traces that astronomers can piece together to determine the gases of the exoplanet’s atmosphere.” — NASA

Read the full articles here:

As said in the articles, this is the strongest evidence humans have ever had for life on other planets. If this is true, it completely shifts our perspective. It means life in our galaxy is not as rare as we thought it would be—it might even be inevitable.

I’ve seen Floppy Disks, Smart Phones, Quantum Computers, Artificial Intelligence and now Aliens. What an era to live in. What a time to be alive.

“The oldest and strongest emotion of mankind is fear. And the oldest and strongest kind of fear is the fear of the unknown.”

First things first, let’s ask this question: What are we actually looking for?

When we stare into the void, we need some kind of framework to make us search better. We need to identify the universal rules that intelligent species follow. To do that, we compare them to ourselves, looking at the energy we consume and the work that we have done.

The Kardashev Scale

This framework ranks hypothetical civilizations based on their energy consumption and technological reach. It scales into different categories:

Type 1: Is able to harness and use the available energy of their entire home planet.
Type 2: Is able to harness and use the available energy of their star and planetary system (like building a Dyson sphere to capture starlight).
Type 3: Is able to use and control the available energy of their entire galaxy.
Type 4: Is able to use the available energy of multiple galaxies, mastering the visible universe.

Here is the catch: If there were any Type 3 or higher civilizations, we would have noticed their massive energy signatures by now. After all, a civilization that spans an entire galaxy couldn’t hide, and they certainly couldn’t have missed us. So this means any type of life outside our solar system, including whatever might be in the oceans of K2-18b, can only be Type 1 or Type 2.

But if the universe is this old, and life is this common, where are they?

“Two possibilities exist: Either we are alone in the universe or we are not. Both are equally terrifying.”

The Fermi Paradox

Enrico Fermi grasped a deeply unsettling mathematical reality: any civilization with a modest amount of rocket technology and an immodest amount of imperial incentive could rapidly colonize the entire galaxy within a few million years. On a cosmic timescale, that is practically overnight.

We look up at a sky filled with billions of stars, many much older than our own sun. Yet, we hear absolutely nothing. Just radio static. I mean, if we’re not alone, why is it so deafeningly quiet out there?

The Great Filter

One of the most well-known, and most dread-inducing, explanations for this paradox is that there is a “Great Filter.”

The theory suggests there might be a hard step in the evolution of life that’s extremely rare—either before intelligent life arises or still ahead of us. In other words: Some step between the start of life and becoming a galaxy-spanning civilization acts as an impossible wall. It is so hard to get past that almost no one makes it.

In our own civilization, this leads to two scenarios:

1. The Filter is Behind Us: If the hardest step is already behind us—like the initial leap from simple to complex life—then we might be incredibly lucky. We are the survivors. Maybe we’re one of the only ones to make it this far.
2. The Filter is Ahead of Us: If the hardest step is still ahead—like surviving our own technology, artificial intelligence, nuclear war, or climate collapse—then finding life on K2-18b is bad news. It would mean life starts easily but most civilizations self-destruct before reaching the stars, and we might too.

The Ultimate Unknown

So, what does this all mean for us? Finding signs of an ocean world 124 light-years away doesn’t just change science; it fundamentally changes how we view our place in the universe.

When I look up at the night sky now, I don’t just see the quiet mechanics of stars and distant planets. I see a massive, silent test. If something is out there on K2-18b, breathing in their hydrogen-rich atmosphere, I wonder if they look at their own sky and ask the exact same questions. Until we know the answer, we just have to keep training our models, building our future, and hoping the Great Filter is a hurdle humanity has already cleared.