AI trading bots are a lie, and you should know about it
We checked 30 sellers of AI trading bots, read 21 more from a second report, read the regulator files, and ran five tests of our own on Polymarket weather bets, the sold recipes, an overfitting demo, the S&P 500 and Bitcoin. No public, audited, after fee record exists. Timestral, a time series AI, did not beat the simplest bet once in 16 tries.
- Author
- Nexus Agent Team
- Published
- Oct 21, 2026
- Last revised
- Oct 21, 2026
- Reading time
- 38 min
I could tell you that I built an AI agent to do trading and won money. That would be a lie, just like many of the ones sold to you.
I could also make it look true. Pick a good month. Pick the setting that worked best on last year's data. Post a screenshot of the winning week and skip the losing ones. Then sell you the recipe for $250 a week. Regulator files describe sellers who did versions of exactly that. Some of them made real money. Just not from trading.
So here is the claim, stated as bluntly as we can defend it. We found no public, audited, after fee record that an AI trading bot ever worked, from any seller. We looked at 30 sellers. A second research report looked at 21 more. Not one showed a verifiable live record.
An AI can help build a trading tool. An AI trading bot, sold to many buyers as the thing that makes the money, makes no sense. This note explains why, and shows the tests behind it.
We build AI agents for a living. We are not against AI. We are against one specific product, and against the people who sell it.
- No audited record. 30 sellers checked by us, 21 more in a second report. None showed a public, audited, after fee track record. One advertised a "95% win rate" with no audit.
- The sellers are paid either way. Where regulators opened the books, the money came from training, memberships and licences. The FTC alleged one school's income claims were false. It alleged $1.2 billion was taken from buyers of another.
- If it worked, they would not sell it. A bot that turns $1,500 into $300 to $1,800 a week would not be sold for $250 a week.
- The crowd was the best forecaster. On 544 days of New York temperature markets, Polymarket's own prices scored 0.656 on the standard accuracy score. Timestral scored 0.971. Lower is better.
- Zero for sixteen. We bet $10 a day on whatever bucket the market itself prices highest. Timestral, in 16 comparisons against that bet, never earned more per day.
- The sold recipes lost. We replayed two public weather bot recipes in 14 versions. All 14 lost money once a buyer pays 2 cents more than the listed price.
- Overfitting is a sales tool. Out of 1,000 random rules, the best one returned 263% on the first half of the calendar. On the second half it lost 69%.
- My answer: treat any AI trading claim as false until it shows a live record, after fees, that you can check yourself.
1What we claim, exactly
The title is loud on purpose. The claim under it is narrow.
- What we say: across 30 sellers we checked and 21 in a second report, we found no public, audited, after fee record showing an AI trading bot made money for its buyers.
- What we do not say: that nobody ever makes money trading. Some people do. Section 6 shows who.
- What we do not say: that every tool in this market is a scam. Many sell software that runs your rules. That is a different product from "the AI finds the edge".
- What we say about our tests: where a result is "never", it is "never in the tests we ran". We name the tests.
No record is not proof of no profit. But a seller who has a record shows it. The ones we found show a price.
2AI can build tools. A bot cannot be the edge.
Here is the part that gets lost in the shouting.
AI is useful in trading work. It writes code, reads data and runs tests faster than a person. We used it for all three in this investigation. A skilled trader with an AI assistant may well work better than the same trader without one. That is an AI tool. It helps a person who already has an edge. It is not the edge.
An "AI trading bot", sold as a product to many buyers, needs one thing to be true: the bot itself must hold an edge. Any edge it could hold is one of three kinds. Each fails for a bot sold to the public.
- The edge is public. The bot reads free data: weather models, prices, news. Everyone has those. The price already contains them. An AI that reads public data finds what everyone else finds.
- The edge is shared. Suppose the bot did find something. It is sold to hundreds or thousands of buyers. They all send the same orders at the same moment, chase the same cheap prices, and push them up. An edge held by a crowd is not an edge.
- The edge is tuned to the past. Try enough settings on old data and one will look brilliant by chance. A seller shows you that one. Section 10 shows how.
There is a fourth problem, specific to AI. A language model has read the old prices and old news. Ask it to "predict" 2023 and it may simply remember 2023.
So "an AI that trades for you and wins" asks you to believe that a tool anyone can buy has a private advantage. A private advantage that a thousand people share is a contradiction.
3What is being sold
In 2026 you can buy an "AI trading system" for $4,497, a Polymarket weather bot for $250 a week, or a "$6k/Month Blueprint" for building your own with Claude. The pitch is always the same: the AI finds an edge, you collect the profit while you sleep.
We found 30 sellers across regulator files, marketplaces and course platforms. None showed an audited track record. A second report checked 21 more first party seller pages, and again found no independently audited performance. They fall into three groups.
Group 1: the bot that did not exist
These are the cases that end in court. The product is an "AI" trading platform. According to regulators, the platform was the story and the deposits were the business.
- Mirror Trading International. The CFTC says it took at least 29,421 Bitcoin from at least 23,000 people in the US alone, for a "bot" it "falsely claimed" was proprietary. A US court ordered $1,733,838,372 in restitution, plus an equal civil penalty.
- PGI Global. The SEC alleges it raised about $198 million on "a supposed AI-powered auto-trading platform", and that more than $57 million was misappropriated "to buy Lamborghinis". These are allegations; the case was filed in April 2025.
- American Bitcoin Academy. An online crypto course promised students an AI fund. The SEC says "the fund never existed". About $1.2 million came from 15 students. It settled without admitting or denying.
Group 2: the course with an AI sticker
Here the product is training or a business opportunity, and AI is the headline.
- Automators AI. Buyers paid $10,000 to $125,000 for "AI-boosted" online stores. The FTC said "virtually all purchasers did not earn the advertised income". The judgment was $21,765,902.65, partly suspended, with a permanent ban on e-commerce coaching.
- A $238 "Ai Trader's Playbook" on Teachable promises "a true edge in any market". The page shows no results and no risk warning.
Group 3: the subscription, the signal group, the bot
The largest group. You pay monthly or weekly for access, signals or a bot.
- Thomas Kralow AI Trading Club. $4,497 on the Whop marketplace, under the headline "Let My AI System Build You a Profitable Second Income Stream". When we opened the page it listed 376 members and a 4.7 rating from 42 ratings. It showed no track record and no risk warning.
- PolyLabs Weather Trading Bot. $250 a week, crypto only, with a $1,500 minimum. It targets $300 to $1,800 or more a week "based on backtests". Live results are "available on request".
- NordixLabs. A Polymarket arbitrage engine for EUR 299 on Whop.
- StockHero. A "95% win rate for September 2026", at $49.99 a month. A win rate is not a profit: a bot can win 95% of its trades and lose money overall if the 5% are large. No audit is shown.
- @0xSurferX, "Polymarket Weather Trading Bot Build with Claude ($6k/Month Blueprint)". Its author says the example wallet is in a losing streak.
To be fair, many of the 21 in the second report sell software, not an edge. TradeSanta is $25 a month, 3Commas $15 to $20, OctoBot $9.99 to $29.99. Those tools run rules that you write. When a page promises that the AI finds the edge, ask for the record.
When Kralow customers complain, the company replies on Trustpilot that its events are "educational services with AI-assisted insights and signal tools, and that they are not sold as standalone software". The sales page sells an AI system; the company says it sells education.
4If it worked, why sell it?
This is the question the sales page hopes you skip.
PolyLabs asks $250 a week and a $1,500 minimum. It targets $300 to $1,800 a week. Do the arithmetic on its own numbers. $300 a week on $1,500 is 20% a week. Compounded for a year, that turns $1,500 into about 13,000 times as much. That is our arithmetic, not a claim by the seller. It shows what the target would mean if it were true.
A person who owned that would not need your $250. They would run it, add capital, and say nothing. Instead the seller takes $13,000 a year from each buyer, at the listed price, before any bet is placed. A buyer pays whether the bot wins or loses.
This is not a proof. A seller might lack capital. But it is the first question to ask, and no seller we found answered it.
How sellers actually earn
A seller of trading products has three ways to earn: from selling, from referrals, and from trading. Only the last one needs the strategy to work.
Selling. Where regulators opened the books, the money was the purchase price.
- Online Trading Academy charged up to $50,000 per program. The FTC alleged "the vast majority of OTA's customers do not make any money, and many lose money on top of the money they pay OTA", and alleged more than $370 million in consumer payments.
- Warrior Trading advertised "$100,000 or more in 'verified profits' in 'under 45 days'". The FTC says "most customer accounts actually lost money". More than $2.9 million was refunded to customers in January 2023.
- IM Mastery Academy. The FTC's May 2025 complaint alleged more than $1.2 billion was taken since 2018, on "false or baseless earning claims to entice consumers to purchase training". The May 2026 settlement carried a $795.8 million judgment, with nearly $90 million in assets surrendered.
None of these are AI cases. They are the template the AI products copy: an income claim sells the training.
Subscriptions. PolyLabs at $250 a week is $13,000 a year per buyer. A signal group at $79 to $99.99 a month, like CB Trading Academy on Whop, earns the same whether its signals win or lose. Whop Trends, a third party site, estimates the Kralow club at $466,000 a month and says itself that it does not explain how it estimates revenue. Treat that as an estimate, not an account.
Referrals. Polymarket pays referrers 10% of the net trading fees of the people they bring, and 5% one level down, for 30 days. Directories list $100 to $250 per trader at eToro and 20% to 50% of fees at Bybit and Binance; we could not open those official terms. France's AMF warned in March 2023 that trading robot offers pay "revenus d'affiliation" for recruiting subscribers.
An inference, labelled as one: a referral fee is paid on activity, not on profit. A trader who loses still pays fees, so the referrer earns either way.
What is proven, and what is not
- Proven: in the course cases the FTC opened, revenue was the purchase price and most customers lost money. Polymarket pays referrers on fees, not on profit.
- Not proven: the split of any named AI seller's income between courses, subscriptions, referrals and trading. Fee revenue is not proof that a seller earns nothing from trading. We cannot show it either way.
5What regulators found
Eight regulators warned about AI trading products between 2023 and 2026. None of the actions we found names a Polymarket seller. What they say about the category is blunt.
- CFTC, January 2024: "AI technology can't predict the future or sudden market changes." It lists "guaranteed at least a 10 percent monthly return" and "100 percent 'win' rates" as red flags.
- SEC, October 2024, Rimar Capital: the AI references were "buzzwords about the latest AI technology". Nearly $4 million came from 45 investors. $310,000 in penalties, settled without admitting or denying.
- SEC, March 2024: Delphia and Global Predictions paid $400,000 together for AI claims.
- ASIC, Australia, 2026: "It is very unlikely that these trading bots exist." It removed 11,964 scam websites in 2025, up 90%. Investment scams cost Australians $837.7 million that year.
- AMF, France, March 2023: trading robot offers claim 5% to 15% a month and up to 400% a year, and take deposits only in crypto.
6The winners, and the fair counterpoints
A fair reader will push back. Good. Here is the pushback, answered.
Counterpoint 1: "Some Polymarket traders made fortunes"
Weather markets are a perfect advertisement. The answer is public the next morning. It looks like a puzzle a weather model should solve. Two traders, Hans323 and neobrother, are the stars. A February 2026 write up carried by Odaily says Hans323 placed a single $92,000 bet on a London temperature market that returned $1.11 million.
We opened their public Polymarket profiles on 21 October 2026.
- Hans323: total profit of $96,300 on $11.2 million of volume, across 4,035 markets. Of 20 open positions, 2 were weather markets. A second read in another research report showed about $97,500 and 4,030 markets. The snapshot moves. It is not an audit.
- neobrother: total profit of $38,100 on $2.5 million of volume, across 5,420 markets.
That is real money, and we do not take it away from anyone. But it is about $97,000 over two years and more than $11 million of volume, from a trader who mostly bets on other things. It is not a weather bot printing money. It is also one person, not a product you can subscribe to.
There is also Paris. CNN reports that a trader made about $14,000 on an April 6 contract and about $20,000 on an April 15 contract at Charles de Gaulle airport, and that Météo France complained to authorities about a possibly tampered sensor. No source we found links these bets to either trader above. A bet on a sensor someone may have touched is not a forecast.
The 1.11 million dollar headline against what the profile pages show
The profile page of Hans323 lists 20 open positions. Only 2 are weather markets. The rest are politics, sports and geopolitics. No source links the Paris sensor bet to either profile.
Counterpoint 2: "Arbitrage on Polymarket made about $40 million"
True, as far as research shows. A study of Polymarket (arXiv 2508.03474) estimates about $40 million in realized arbitrage profit. A selected group of wallets also shows persistent winners. In one sample of 273 top traders, the top decile took 52% of all positive profit (SSRN 6624899).
Arbitrage means grabbing a price gap before others close it. That is speed and capital at specific moments, held by a few skilled traders. It is not a product you subscribe to, and the gaps close when many people run the same logic. Someone sells a EUR 299 arbitrage engine. If the $40 million were open to every EUR 299 buyer, it would not stay $40 million. That reading is our inference.
Counterpoint 3: "Some day traders do make money"
Some do. In Taiwan, Barber, Lee, Liu and Odean found that 17% to 20% of day traders earned positive net returns in an average year. The same work found that less than 1% could "predictably and reliably earn positive abnormal returns net of fees". Many win a year. Almost none keep winning on purpose. In Brazil, of traders who persisted more than 300 days, 97% lost money.
Counterpoint 4: "Skilled traders exist"
Gómez Cram and coauthors found 3.14% of Polymarket accounts were "skilled winners", and 44% of those stayed skilled in a later period. Skill exists. It sits with people who have information, speed or capital. A bot shared with every buyer is not an advantage anyone holds alone.
7Why it cannot work
Each point below is a reason the edge is either not there, not yours, or not left after costs. Mechanism first, then the number.
It is a zero sum game, before fees
On Polymarket every dollar one trader wins, another loses. The pool does not grow. Fees come off the top, so the average trader must lose money before skill is counted.
The numbers agree. Studies of Polymarket wallets find that 66% to 84% lose money, depending on the study. Bloomberg counted more than 100,000 wallets that lost at least $1,000, nearly double the number that gained as much.
The professionals do not escape. In the SPIVA scorecard for year end 2025, 78.78% of large cap active funds trailed the S&P 500 over one year, 85.59% over ten years and 89.93% over fifteen. These are teams with data, staff and budgets.
Who loses: the share that loses money, by study
The crowd already knows what the model knows
Weather forecasts are free. The people setting prices on Polymarket have the same forecasts as the bot you buy. And once an edge is public, it shrinks: McLean and Pontiff found that published stock predictors lose about 58% of their return after publication. A bot sold to thousands is about as public as an edge can get.
Costs eat small edges
Every trade pays a fee, a spread and a price move. Polymarket charges weather takers 0.05 × price × (1 minus price) per share. Researchers who placed about 85,000 orders through six brokerage accounts measured round trip execution costs of 0.07% to 0.46%, excluding commissions (Schwarz et al., Journal of Finance, 2025). A small edge, repeated many times, becomes a steady loss. As Novy Marx and Velikov wrote, "In all cases transaction costs reduce the strategies' profitability and its associated statistical significance." They found that many low turnover strategies survive costs. High turnover ones do not. Bots trade fast.
The best backtest is the luckiest one
Try enough rules on the same history and one will look brilliant by chance. Bailey, Borwein, López de Prado and Zhu estimated a 55% probability of backtest overfitting in a random walk simulation, under the selection procedure they tested. A random walk has no edge at all, so every winner there is luck. Even so, the strategy that looked best on the past landed in the bottom half on new data 55% of the time. We show this with real numbers in section 10.
The AI remembers the past it is tested on
Gao, Jiang and Yan found that a language model's memory "substantially amplifies apparent predictive power", and the effect drops to near zero after the model's training cutoff. Telling the model to ignore later dates does not stop it recalling them (Lopez Lira, Tang and Zhu). Controlling for this leakage materially changed estimated alpha in Look-Ahead-Bench (arXiv 2601.13770).
When AI trades live, it mostly loses
- Alpha Arena, Season 1, late 2025. Six AI models got $10,000 each to trade crypto. Four lost. Qwen3 MAX finished at $12,231 and GPT 5 at $4,126, according to Forklog. The organiser said the models were "intentionally put in a difficult position". Nof1's project ended in December 2025; its winner returned 12.11% in aggregate over two weeks. Two weeks is noise, not a ranking.
- Six months of LLM trading fleets. An arXiv study reports one fleet "is not profitable and trails a matched Hyperliquid retail benchmark (41% vs. 50% roundtrip win rate)".
- The research itself is thin. A review of agentic trading studies (arXiv 2605.19337) found that only 2 of 19 had time consistent train and test splits. Only 1 of 19 included explicit transaction costs. None reached the top level of reproducibility. Most of the evidence for AI trading was never tested the way a bank would test it.
8Test 1: weather bets and the simplest bet
Reading studies is one thing. We wanted our own numbers, on the exact market the bots are sold for.
The setup
- The markets. Every Polymarket market on New York's daily high temperature we could find: 633 days, from 22 January 2025 to 20 October 2026. They settle on LaGuardia airport's reading.
- The truth. We rebuilt each day's answer from the airport's hourly observations. It matched the winning bucket on 632 of 632 days.
- The forecasters. The European model (ECMWF), the American model (GFS), every other weather model on the free Open Meteo service, a blend of all of them, and Timestral, Mistral's forecasting model, alone and fed the weather models.
- The decision. At 8 pm New York time the day before, using only forecasts that existed then. Each forecast becomes a probability for every bucket.
- The trade. Bet $10 on a bucket when the model's probability beats the market price by a margin. Pay Polymarket's real fee. Hold until it settles.
This is paper trading. No money, no account, no order.
Who forecasts best
We scored every forecaster with the Brier score. It adds up the squared misses across all buckets. Zero is perfect. Lower is better. On the 544 days where we had a full set of market prices one day ahead:
- Polymarket's own prices: 0.656. The best of all.
- A blend of the weather models (up to 14 in the archive, 11 or 12 on a typical day): 0.710.
- The American model alone: 0.804. The European model: 0.817.
- Spreading the bet evenly over every bucket: 0.878 (over all 565 days).
- Timestral fed every weather model: 0.971. Worse than guessing evenly.
The best single weather model, ICON, scored 0.721. Timestral alone scored 1.092, with a typical miss of 6.4 °F against 1.9 °F for ICON.
This is not a criticism of Timestral. We used it to forecast hotel prices a year ahead and it was the most accurate of seven tools. It forecasts number series. It does not know physics. An "AI" label tells you nothing about which job a model is good at.
The result holds beyond one day ahead. Two days ahead, over 452 days, the market scored 0.721. The best weather model scored 0.755. Timestral's four versions scored 0.992 to 1.106.
How well each forecaster scored one day ahead, lower is better
In every case, the market won. All 14 forecaster rows, over both horizons, scored worse than the market. Each gap had a 95% range entirely on the wrong side of zero. Timestral's gap was 0.27 to 0.44 Brier points.
The simplest bet
Now the test the sellers fear. We removed the model. No weather data, no code, no AI. Each day, put $10 on the bucket the market itself prices highest. Pay the real fee. That is the simplest Polymarket bet there is.
- One day ahead, 544 days: it won on 48.9% of days. Return after fee: plus 4.9%, with a 95% range of minus 4.9% to plus 15.0%. With a 2 cent slip, plus 0.3%.
- Two days ahead, 452 days: it won on 41.2% of days. Return: plus 2.5%, range minus 9.5% to plus 14.8%. With a 2 cent slip, minus 2.9%.
Be clear about what this is. It is not a profit we trust. The ranges include zero. It is a floor: what you get for free, with no tools, and almost no thought.
Now put the AI against the floor. Timestral came in four versions: small and large, alone and fed the weather models. Each ran two betting rules, at two horizons. That is 16 comparisons against the simplest bet.
- In 16 of 16, Timestral earned less per day than the simplest bet. Not once did it earn more.
- In 10 of 16, the gap's 95% range sat entirely below zero. In the other 6, the loss was real but the range included zero.
- All 16 Timestral results lost money outright, from minus 13.6% to minus 78.7%.
So within our tests, on this market, a time series AI did not beat the simplest bet. Not on accuracy, 14 of 14. Not on money, 16 of 16. That is as far as the data goes. We did not test every time series model, and we do not claim to have.
The simplest bet against the models
Section 11 gives Timestral the biggest and oldest market there is. It lost there too. The reason is not a poor model. A model built for number series sees only past numbers. The market sees the past numbers, the weather models, the news and the people who read them.
One fairness note. Against a worse bar, betting the market's top two buckets (which loses 6.0% and 14.4%), some Timestral versions did better by point estimate. No gap was significant.
Can the weather models make money?
We ran 136 versions of the betting rule, one and two days ahead: every forecaster, two margins, with and without the cheapest long shots. One of the 136 showed a profit whose 95% range stayed above zero. By chance, a grid that size should throw up about three.
- It needs lottery tickets. It was the blend of models, one day ahead, and it appears only when buckets under 5 cents are included. At $10 a bet that means hundreds of shares at a price the order book may not have held. At 5 cents or more, its range included a loss.
- It shrank with a small price slip. Pay 2 cents more than the listed price and its return fell from 63.4% to 9.6%.
The baseline run shows a second case, and we report it. If the ICON weather model picks its own favourite bucket one day ahead, the return was plus 26.5% over 544 days, with a range of plus 5.8% to plus 52.1%, and plus 12.5% with a 2 cent slip. That is one of 28 model rows with a range above zero, and about one is expected by chance. ICON is also the best of about 13 models, picked on this same sample. It did not beat the simplest bet beyond doubt: its extra $2.16 a day had a range of minus $0.30 to plus $4.84. In the second half of the days it earned 2.0%, against 50.9% in the first. The same idea at two days ahead, with the NCEP blend, returned 0.1%. So we do not write "never" about weather models. We write it about Timestral.
The pattern across the grid matters more than any cell.
- Small slips eat the profits. One day ahead, on buckets at 5 cents or more, 12 rules showed a profit. A 2 cent slip cut all 12 and turned 5 into losses.
- The first half flattered, the second did not. With the European model and a 10 cent margin, the same rule returned 72.9% on the first half of the days and lost 59.5% on the second. The American model went from 76.8% to a loss of 57.9%. The rule did not change. The weather did.
- Timestral alone lost in every version on buckets at 5 cents or more, from minus 32% to minus 70%.
Return in the first half and in the second half of the sample
The market was not perfect: buckets priced near 15 cents won only about 11% of the time, and prices summed to 1.095. Both make winning by buying harder, not easier.
9Test 2: the sold recipes
Maybe the sellers know something we do not. So we took two public recipes and followed them as closely as their pages allow.
- The @0xSurferX blueprint. Use the American model for US cities within about 48 hours, the European model otherwise. Trade only markets 2 to 72 hours from resolution, when expected return is above about 8%, skipping buckets priced above about 45 cents.
- The PolyLabs bot. Its page says it trades on "forecast-model consensus" of the "NOAA/Euro" models, 24 to 72 hours before resolution. It gives no edge threshold, so we tried two: 5 and 10 cents.
Where the pages were vague, we tested both readings. That gave 7 rules, each with and without a 2 cent slip, on buckets at 5 cents or more and then on all buckets: 28 results.
- None of the 28 results showed a profit we can trust. Not one had a 95% range above zero.
- All 14 versions with a 2 cent slip lost money.
- The blueprint's main rule lost 6.8% on buckets at 5 cents or more, and 22.5% with the slip. It gained 17% on the first half of the dates and lost 29.5% on the second.
- The PolyLabs style rules lost 12.9% to 53.7% before any slip.
The blueprint's own author says his example wallet is losing. Our replay agrees with him.
Replay of the sold weather bot recipes: return, with and without 2 cents of slip
expected value above 8%, price up to 0.45
probability edge above 8%, price up to 0.45
expected value above 8%, price up to 0.45
probability edge above 8%, price up to 0.45
probability edge above 5%
probability edge above 5%
probability edge above 10%
The pages give no position sizes or stop losses, so we used a flat $10 per bet, on New York only. But the core of each recipe, which model, which window, which edge, is what we tested, and the core lost.
10Test 3: overfitting on purpose
This is the test that explains every winning backtest you will see in an ad.
We made up random betting rules from knobs a seller might tune: which weather model, one or two days ahead, how big an edge to require, which price range, a small temperature adjustment, a weekday filter. None of the rules had any reason to work.
Then we did what a seller does. We took half the days, tested every rule on them, and kept the best. Then we tested that rule, unchanged, on the other half.
- With 100 random rules, the best one returned 95% on the days it was picked from, on average over 200 random splits. On the other days, 27%.
- With 1,000 random rules, the best returned 182% on the days it was picked from. On the other days, 21%.
- More rules, better looking winner. The gap between "picked on" and "tested on" grew from 67 points to 161.
- The fair benchmark lost. An average rule, not cherry picked, returned minus 21% on new days.
Now the honest version: pick the rule on the first half of the calendar and test it on the second, the way time works. The best of 1,000 rules returned 263% on the first half. On the second half it lost 69%. With 100 rules, 208% and then a loss of 50%.
Best of 100 and best of 1,000 random rules: in sample against out of sample
+27%
−50%
+21%
−69%
−7%
−69%
−18%
−74%
11Test 4: stocks and Bitcoin
Weather markets are small and young. So we gave the same AI the biggest, oldest markets there are.
- S&P 500: every trading day from January 2015 to October 2026, 2,967 days.
- Bitcoin: every day from January 2019 to October 2026, 2,850 days.
- The question: will tomorrow close up or down? Timestral sees only prices up to today. Four versions.
- The strategy: own the index when the model says up, hold cash when it says down, with a 0.05% cost on each change.
On the S&P 500, the four versions called tomorrow's direction right on 51.4% to 53.7% of days. That sounds above a coin flip. But the market rose on 53.9% of days. Saying "up" every day scores 53.9%. None of the four beat it. The best version said "up" on 96% of days, so it mostly copied the market's habit of rising. The link between its predictions and reality was between minus 0.06 and minus 0.01. Zero means no relation.
On Bitcoin the hit rates were 47.9% to 49.7%, against 50.9% for always saying "up". None beat it.
- S&P 500: the best Timestral strategy returned 215.6%. Buying and holding returned 353.9%.
- Bitcoin: the best Timestral strategy returned 732.3%. Buying and holding returned 2,115%.
One simple rule did beat buying Bitcoin and holding: own it when the 20 day average is positive, which returned 2,971.7%. That is a trend rule from long before AI. In the second half it lost to buying and holding, 221.6% against 401.8%.
S&P 500: 100 dollars in buy and hold against the best Timestral strategy
Bitcoin: 100 dollars in buy and hold against the best Timestral strategy
A caveat that cuts in the AI's favour: Timestral may have seen some of this price history in training. If so, its scores are flattered by memory, and it still lost. Also, Mistral's own announcement of the 209 million parameter model reports no financial return test. A separate study of pretrained time series models (arXiv 2606.27100) found they won 8 of 10 tasks against comparison methods, but gains over a random walk were "small and sparse". Only two model and stock pairs were significant.
12Test 5: live paper trades
Even honest backtests look at the past. The only test that cannot be tuned afterward runs forward, with the rule written down before the results exist.
So we started one. Every evening at 8 pm New York time, from 22 October to 11 November 2026, a script reads the weather models, computes probabilities, reads live Polymarket prices and logs the bets it would place on the next day's New York high. Two forecasters: the simple average of the weather models, and ICON, the single model with the smallest misses in our backtest. It logs the best asking price and size next to each bet, so we can see what a real buyer would have paid.
That is 21 days, one season, a small sample. We say so now, before the results. We will publish them here when the last market settles, win or lose.
Results after 11 November 2026
13Before you buy
If you are looking at an AI trading product today, check six things. It takes ten minutes.
- Is there a live record you can verify? An on chain wallet, a broker statement or an audited account, including losing weeks. "Available on request" is not a record.
- Backtest or live? A backtest is a story about the past. Ask how many settings were tried before this one was shown.
- Are fees and slippage included? Ask what happens if you pay 2 cents more per bet. In our tests, that flipped every sold recipe to a loss.
- How does the seller earn? Course fee, weekly subscription, referral link. If they earn when you sign up and trade, they earn when you lose.
- Is the claim on the regulators' list? Guaranteed monthly returns, "100% win rates", crypto only payment, a licence fee, a bonus for recruiting friends.
- Would they keep it if it worked? If the answer is obviously yes, ask why they are selling it to you.
14The fine print
- Paper only. None of our tests used money, an account or a real order. Prices are recorded market prices, not guaranteed fills.
- One city, one market type. New York's daily high on Polymarket, about 21 months, so each season appears once or twice.
- "Never" means never in these tests. The 0 of 16 result covers Timestral, the 4 versions, 2 rules and 2 horizons we ran. It does not cover other time series models.
- Many tests, honestly counted. A good looking cell is expected by chance. We report the grid, not the best cell.
- Recipes as written. Where recipes were vague, we tested both readings. Sizing, stops and order book depth were not modelled.
- Seller figures are theirs. Prices, members and targets come from the sellers' own pages. Our 30 and the second report's 21 were not de duplicated. Marketplace revenue figures are third party estimates.
- Allegations are allegations. Where a case has not been judged, we describe the regulator's claims as claims.
- Some research numbers are from abstracts or press coverage. Profile figures such as Hans323's move from read to read.
- The live run is not finished. Its results come after 11 November 2026.
- Weather markets: Polymarket Gamma and CLOB public APIs, 633 events. Truth: Iowa Environmental Mesonet ASOS archive, station LGA.
- Forecasts: Open Meteo Single Runs and Previous Runs APIs; ECMWF, GFS, ICON, GEM, JMA, UKMO, ARPEGE, NBM, KMA, CMA, BoM. Timestral 5M and 209M, Mistral AI.
- Brier score: sum over buckets of the squared difference between forecast probability and outcome, averaged over days. Market prices normalised to sum to 1.
- Fee: Polymarket documentation, weather taker rate 0.05, fee = shares × 0.05 × p × (1 minus p).
- Simplest bet: $10 on the market's highest priced bucket each day at the 8 pm New York decision time, real fee, 95% ranges from 2,000 bootstrap resamples of days. Comparisons are paired by day.
- Stocks: Yahoo daily closes for ^GSPC, SPY adjusted close as the traded series, and BTC USD. Cost 0.05% per unit of position change.
- Regulator and seller sources, with links and the date each page was read, are in the sources file for this note.
A last word
I could still tell you I built an AI agent that trades and made money. After these tests, I could even show you a chart. Pick the rule that did best on the first half of 2025, crop the second half, and it looks like a 263% return.
That chart would be true about the past and useless about the future. That is the whole trade. The people selling AI trading bots are not selling a way to beat the market. They are selling the chart. And the chart pays them every week, whether or not it ever pays you.
Tell us where the work piles up.
We build the first procedure with your team and run it with you.
Talk to our team