3. Formula
The score is the average of four questions, and each question is the weighted average of its signals. Every signal has a weight w of 1, except AI-built sites at 0.25, because its history is short. Counting questions equally stops a busy area from outvoting a quiet one.
How each signal scores
Here clamp limits a score to between 0% and 100%, x₀ is the value in November 2022 or when its data starts, and t is the current month.
Fell since the start
s=clamp(−100log10x0xt) Prices after inflation, revenue and software spending against GDP, security holes, bugs and incidents per git push, job openings, outsourcing revenue after inflation and Stack Overflow questions. Falling to a tenth scores 100%, and any rise scores 0%. Token prices use 50 in place of 100, so they need to fall to a hundredth.
Why this goal: A round number and our judgment: a tenfold fall is too big to put down to normal movement. Token prices fall much faster than most prices, so a tenth would be too easy.
Grew since the start
s=clamp(100log10x0xt) Code activity, no-code revenue and new tech businesses. Growing tenfold since November 2022 scores 100%.
Why this goal: A round number and our judgment, matching the tenfold fall above.
A direct share
s=100⋅sharet Coding benchmarks, AI pull requests, the programming share of AI, developer sentiment and AI company hiring score the share itself.
Why this goal: No goal chosen, because a share stops at 100% by itself.
Length of task
s=clamp(100ln120000lnmt) The longest task, in minutes, that the best model completes half the time, up to a working year of 2,000 hours.
Why this goal: Our judgment: a working year is about what one engineer delivers alone. It is not a measured threshold.
Cost of a hard task
s=clamp(10033−log10ct) The cheapest near-top run on FrontierSWE, from $1,000 a task at 0% to $1 at 100%.
Why this goal: Our judgment: $1,000 is roughly a day of an engineer's time, and at $1 the cost of a task stops mattering.
Growth from a recent start
s=clamp(1003log10(xt/x0)) AI and coding app tokens, and sites on AI builders, against their first month. Growing 1,000 times scores 100%.
Why this goal: A round number and our judgment: these start from a small base, where tenfold comes quickly.
Output above its trend
s=clamp(100log2TtWt) Output per tech worker W against its 2015 to 2019 trend T. Doubling that trend scores 100%.
Why this goal: Our judgment: doubling means one worker producing what two did on the old trend.
Jobs against the market
s=clamp(100(1−J0/M0Jt/Mt)) How much software postings J have lost against all US postings M since ChatGPT launched.
Why this goal: No goal chosen: 100% means software postings have disappeared against the market.