What goes in
- Headlines. The collector reads the public feeds of crypto outlets and a few regulators. It keeps the title, the link, the outlet name, and the time. It does not keep or show article text or images.
- Prices and market data. Prices: CoinPaprika and exchange public APIs. Market data: Alternative.me, mempool.space, Hyperliquid, DefiLlama. Tokens: DexScreener, GeckoTerminal, GoPlus. Price history: Coinbase Exchange public market data.
- Token checks. On-chain facts about a token, such as who can mint or whether an exit swap works, come from public safety sources (GoPlus, honeypot.is, RugCheck) and from the pair data of DexScreener and GeckoTerminal.
Which models run
JEV is a fast typed classifier. The official endpoint is TypeSafe's System One model. It reports the model name jev-latest. We do not pin a version yet, so a provider update can change a rating. We log which provider answers each call. This is shown on the status page.
We also operate two JEV models on our own GPUs, to compare against the official one. That comparison is offline right now. The status page shows agreement figures only when comparison samples exist. A local model never changes what the site shows.
How the impact rating works
JEV reads the headline and answers typed questions. The impact answer has four classes:
- Market moving. A story that could move prices soon, such as a hack, a large fund flow, or a regulator decision.
- Notable. A story that a follower of the sector should know.
- Routine. A regular update or a market wrap.
- Promo. A promotion, a sponsored post, or noise.
The page shows how sure JEV is of each answer as a percent. That number is the confidence of the model in its own answer. It is not a chance that a price moves. A headline that JEV has not judged shows "Awaiting JEV".
Sentiment is the model view of the effect on the asset price: positive, negative, or neutral. It is not a recommendation.
How good are JEV's ratings here?
We asked a larger reference model (deepseek-flash) to rate 240 news items of 7 days with the same questions as JEV. Then we counted how often JEV gave the same answer. This is agreement with a reference model. It does not show how often JEV is right: the reference is another model, not people, and both can be wrong in the same way. Every number has its 95 percent interval. Two plain baselines stand beside JEV. The constant guess always gives the most common answer. The keyword rules were written before any label existed and were never tuned on the sample.
Measured Sep 29, 10:31 UTC. Sample: 240 items, 7 days, English headlines, one collector.
| Question | JEV | Constant guess | Keyword rules | JEV minus keywords |
|---|---|---|---|---|
| Impact, four classes | 70.0% (64.2% to 75.8%) | 57.9% (51.7% to 64.6%) | 40.0% (33.8% to 45.8%) | Clearly better+30.0 points (+21.7 to +37.5) |
| Sentiment | 69.6% (63.7% to 75.4%) | 52.9% (46.2% to 59.2%) | 66.2% (60.4% to 72.5%) | Not clearly better+3.3 points (-5.0 to +11.7) |
| Category | 81.7% (76.7% to 86.7%) | 33.3% (27.5% to 39.2%) | 55.8% (49.6% to 62.5%) | Clearly better+25.8 points (+18.8 to +34.2) |
| Sponsored or not | 96.2% (93.8% to 98.3%) | 95.0% (91.7% to 97.5%) | 95.4% (92.9% to 97.9%) | Not clearly better+0.8 points (-2.1 to +4.2) |
| Impact, two classes (important or not) | 75.8% (70.0% to 80.8%) | 64.2% (57.9% to 70.0%) | 59.6% (52.9% to 65.4%) | Clearly better+16.2 points (+8.8 to +23.3) |
The sample holds more rare classes than the real feed. The column "weighted" corrects for that. The other columns describe the sample. For the sponsored question a constant guess of "not sponsored" already agrees most of the time, because few items are sponsored.
Where JEV is weak
- Sentiment: JEV is not clearly better than the keyword rules. The gap is +3.3 points (-5.0 to +11.7) and the interval includes 0.
- Sentiment: JEV is overconfident. Its mean confidence is 81.6% and it agrees on 69.6%.
- Category: JEV is overconfident. Its mean confidence is 87.1% and it agrees on 81.7%.
- Sponsored or not: JEV is not clearly better than the keyword rules. The gap is +0.8 points (-2.1 to +4.2) and the interval includes 0.
- Sponsored or not: JEV is not clearly better than the constant guess. The gap is +1.2 points (-2.1 to +4.6) and the interval includes 0.
- Impact: when JEV says routine, the reference model agrees 52.5% of the time (routine recall 79.2%).
| Class | Precision | Recall | Items in the reference |
|---|---|---|---|
| market moving | 86.7% | 86.7% | 15 |
| notable | 92.3% | 60.4% | 139 |
| routine | 52.5% | 79.2% | 53 |
| promo or noise | 53.7% | 87.9% | 33 |
Precision is the share of items that JEV put in the class and that the reference model put there too. Recall is the share of the reference items of the class that JEV found.
| Confidence of JEV | Items | Agreement | Mean confidence |
|---|---|---|---|
| under 0.5 | 28 | 28.6% | 43.4% |
| 0.5-0.7 | 86 | 59.3% | 59.5% |
| 0.7-0.9 | 70 | 80.0% | 79.8% |
| 0.9-1.0 | 56 | 94.6% | 94.7% |
| JEV | deepseek-flash | |
|---|---|---|
| Time per news call, median (p50) | 141.5 ms | 766.4 ms |
| Time per news call, p95 | 198.5 ms | 1,041.5 ms |
| Cost per 1,000 headlines | $0.0031 | $0.2212 |
| Cost per day, observed | $0.0013 | $0.0951 |
| Cost per day, at 480 news calls a day | $0.0015 | $0.1062 |
JEV time comes from the production log of the collector, up to Sep 29, 09:53 UTC: 430 news calls in the last 24 hours, 2.2 hours of log. The reference time comes from the test run. The two are not a fair race. The JEV cost counts the text of the headline only (an estimate). If JEV also bills the question text, it is $0.0233 per 1,000 headlines. The reference cost uses the peak list price ($0.1106 off-peak).
Limits of this test
- The reference is another model (DeepSeek Flash), not people. The numbers show agreement with the reference model. They do not show how often JEV is right. Both models can be wrong in the same way.
- The sample is small: 240 items from 7 days of one collector. The intervals are wide for the rare classes. The smallest impact class has 15 items in the whole window.
- The sample is stratified on the JEV impact class, so rare classes are over-represented. Raw agreement, macro F1, and the confusion matrices describe the sample. Only the weighted agreement describes the real mix of the feed.
- All items are English headlines with a feed summary of at most 300 characters. The models did not read the articles. No other language was tested.
- One story can appear in several outlets, so near-duplicate headlines can occur more than once. Inside each class the draw is balanced across outlets, so a big outlet is under-represented.
- The reference model got the same question text as JEV in one JSON call. JEV gives typed probabilities in one call. The two prompts differ, and a different prompt can change the labels of the reference.
- The sponsored class is rare, so a constant guess of false looks good on agreement. Read the macro F1 and the confusion matrix for that question.
- Reference answers that were not valid were left out and counted, never guessed. Problems found: 0.
- JEV latency comes from the production log of the collector on the web box, with parallel calls. The log spans 2.2 hours, and the first hour holds a backlog. The reference latency comes from this run on the server with at most 4 parallel calls. The two are not a fair race.
- JEV token counts are an estimate (characters divided by 4). Chinese public holidays are not modeled in the peak-hour price of the reference model.
How the signal score works
The signal score runs from 0 to 100. It rises with the impact class, with how fresh the story is, and with how many outlets carry it. A story from many outlets scores higher than a single-outlet story of the same class. The score orders the JEV Pulse. It is not a forecast.
What makes a signal
A signal is a public model output that we log once. The kinds in use are: high-impact news, a stablecoin depeg, an extreme perpetual funding rate, and a new DEX pair that carries a risk flag. When we log a signal, we store the price of the asset at that moment. A signal without an asset has no price and no grade.
How a hit rate is computed
- For each signal with a direction and an asset, the tracker reads the price again 1 h, 4 h, and 24 h after the signal.
- A hit means the price moved in the direction of the signal at that horizon. A miss means it moved the other way. A move of exactly zero counts as a miss.
- The denominator for a horizon is the number of signals that have a direction and a recorded price at that horizon. Signals still waiting for their horizon are not in it. Signals without a recorded price are not in it.
- A price counts only if it was read within 30 minutes after the due time. A later price is not used. The cell stays empty. We do this so that a late price cannot flatter or hurt a signal.
- A hit rate shows only when at least 30 graded flags are in the denominator. Below that, the page says "not enough graded flags yet" and shows no rate.
- Next to a rate the page shows a 95 percent interval (a Wilson interval). It shows how much luck could explain the rate. A wide interval means that the sample is small.
- The average absolute move is the mean of the absolute price change over the signals with a price at that horizon. It has no direction.
Why 50 percent is the number to beat, and what the baseline is
A signal points up or down. If it knew nothing, the price would move its way about half of the time, so pure chance gives a hit rate near 50 percent. A hit rate of 54 percent from 40 flags is easy luck. A rate is worth something only when its whole 95 percent interval lies above 50 percent.
The move needs a yardstick too. Every asset moves a little in any hour. The baseline is the typical absolute move of the same asset in a random window of the same length. The scoreboard shows the move after a signal next to it. If the move after a signal is not larger than the baseline, the signal added nothing.
The signal log shows every signal that we logged, including the misses. We do not delete or edit a signal after it is logged. The scoreboard covers the last 300 signals that the collector publishes.
How we test the impact rating against the price
The signals page compares the size of the price move after news with the typical move in a random window. For each JEV impact class it shows the mean absolute move of the assets that the stories name, 5, 15, and 60 minutes after the story, and the typical 60 minute move at any time. It also asks whether market moving stories are followed by larger moves than routine stories, and it shows the p-value of that comparison. A small p-value says that a gap this large is unlikely to come from chance alone. It does not say why a price moved. With too few stories in a class, the page says that it needs more data, names the number that it needs, and shows no p-value.
How the token risk estimate works
Automated risk estimate from public checks and a statistical model. Not a finding of fraud. The estimate has three parts:
- Rules. Each rule checks one fact. Examples: the mint authority is active, liquidity is low or not locked, one wallet holds a large share, the exit tax is high, the pair is very new, or a safety source reports that the token cannot be exited. Each rule that fires is shown as a flag with its source and time, so you can check it yourself. A flag is a fact. It is not a label.
- Classifier note. JEV answers typed questions about the token name and text. From its answers we compute a concern value between 0 and 1 and an impersonation value between 0 and 1. The concern value is the probability of the top answer plus half the probability of the middle answer.
- Model. A statistical model turns the rule results and the JEV answers into a predicted chance. It learned from tokens that we listed earlier and whose outcome we know. The chance is that liquidity collapses or that the token vanishes within one hour, and within 24 hours.
The label is a band of the predicted chance within one hour. The pages show the chance as a band (under 10, 10 to 25, 25 to 50, 50 to 75, or 75 percent or more) and never as an exact number: low risk under 10 percent, elevated risk from 10 to 25 percent, high risk from 25 to 50 percent, and critical risk from 50 percent up. The label means the predicted chance that liquidity collapses or the token vanishes within an hour. Two more states exist. A token that no safety source answered gets the label not checked, because "low" would claim a check that did not happen. While the model is preliminary, every label carries that word, because the model has seen too few real observations. While no model is adopted, tokens show as unscored: the flags stay, and no label is claimed. A critical label also needs at least one flag that names its source. A value that a check did not give never counts as safe.
The label is an automated estimate from public checks and a statistical model. It is not a finding of fraud, and it is not a claim about the people behind a token. Every new token is risky, and a low label is not a safety claim. A label can be wrong. See how to dispute a label. Every checked token has a page with its facts and its outcome. You can check a token yourself.
How the Rug Radar model is tested
- AUC is the chance that the model gives a higher number to a token that collapsed than to one that did not. 0.5 is a coin flip and 1.0 is perfect. The whole interval lies above 0.5, so the model ranks tokens better than a coin flip on this test set.
- Brier score is the mean squared error of the predicted chances. The model is closer to the outcomes than the base rate is.
- Spam cohort. 420 tokens, 4.2 percent of the tokens seen, look like copies made in bulk. They are part of every number on this card.
| Set | Time span | Tokens | Alive | Liquidity collapsed | Vanished |
|---|---|---|---|---|---|
| Train | Oct 01, 23:37 to Oct 02, 11:51 UTC | 5,305 | 4,361 | 801 | 143 |
| Test | Oct 02, 15:55 to Oct 02, 22:01 UTC | 2,601 | 2,005 | 320 | 276 |
The test tokens are all newer than the training tokens. 27 tokens were left out because they had no safety check. Alive means that the pair was there and kept its liquidity.
| Chain | AUC | Tokens |
|---|---|---|
| solana | 0.79 | 2,390 |
| robinhood | 0.67 | 87 |
| Chain only (baseline) | 0.56 | - |
A model that knows only the chain scores 0.56. The fitted model is 0.24 above it.
| Predicted bucket | Predicted average | Observed rate | Tokens | Match |
|---|---|---|---|---|
| under 10 | 2% | 6.6% | 1,104 | The predicted average is outside the interval. |
| 10 to 25 | 13% | 12.8% | 421 | The predicted average is inside the interval. |
| 25 to 50 | 30% | 34.8% | 838 | The predicted average is outside the interval. |
| 50 to 75 | 66% | 57.4% | 61 | The predicted average is inside the interval. |
| 75 or more | 86% | 80.2% | 177 | The predicted average is outside the interval. |
| Rule | Value | Needed | Result |
|---|---|---|---|
| auc pooled lower | 0.782 | 0.65 | Passed |
| auc solana lower | 0.765 | 0.6 | Passed |
| brier margin | 0.038 | 0.005 | Passed |
| test span hours | 6.1 | 6 | Passed |
| test tokens | 2601.0 | 300.0 | Passed |
| train span hours | 12.233 | 24 | Failed |
| train tokens | 5305.0 | 500.0 | Passed |
| When | State | Pooled AUC | Note |
|---|---|---|---|
| Sep 29, 15:35 | unscored | - | the candidate is unscored and may not label: only 35 native training tokens are left (need 150) out of 890 native labelled tokens after the spam cohort and two embargo gaps of 1 hour |
| Sep 29, 16:10 | unscored | - | the candidate is unscored and may not label: only 37 native calibration tokens are left (need 50) out of 1095 native labelled tokens after the spam cohort and two embargo gaps of 1 hour |
| Sep 30, 16:10 | preliminary | 0.86 | the current model is unscored and may not label; the candidate is preliminary |
| Oct 01, 23:02 | preliminary | 0.86 | the candidate Brier score 0.1049 is not worse than the current 0.1187 on the same 2702 recent test tokens (6.3 hours); the candidate is preliminary |
| Oct 02, 23:02 | preliminary | 0.80 | the candidate Brier score 0.1347 is not worse than the current 0.1368 on the same 2601 recent test tokens (6.1 hour); the candidate is preliminary |
The model was tested on tokens that it had not seen in training. The test set is small (2,601 tokens), so the intervals are wide and a result can change with new data. The model predicts that liquidity collapses or that a token vanishes. It does not detect fraud.
How we test the labels
For every token that we list, we store the first price and liquidity. One hour and 24 hours later we read the pair again. A token counts as collapsed when liquidity fell 80% or more, or the price fell 90% or more. It counts as vanished when the pair no longer appears in the data. The radar page shows, for each label and for each horizon, how many tokens there were, how many collapsed or vanished, that rate with its 95 percent interval, and the rate of all listed tokens (the base rate).
The page says that a label collapsed or vanished more often than average only when the whole interval of the label lies above the base rate. If the interval reaches the base rate, the page says "no measured skill yet". A rate shows a dash for a group with fewer than 10 tokens. A vanished token is counted with the collapsed ones, because its market is gone. The page also shows the share of tokens that disappeared within 24 hours, so you can see how much of each rate comes from them.
The first label of a token is frozen when the radar first sees it, so a label never sees the outcome that it is tested against. The page shows a chance band and never an exact number. Until the model is adopted, every label carries the word "Preliminary" or "Unscored".
What we got wrong first
2026-09-29. Our first hand-written rules ranked tokens no better than chance (AUC between 0.46 and 0.56, depending on whether you rank by label or by the stored score; we did not compute an interval at the time). Tokens labelled critical collapsed or vanished less often than the average token. These are historical facts about the earlier version, not about the model on this page. We replaced the rules with a fitted model on the same day.
A round 3 model then reported an AUC of 0.90 on a test window of 58 minutes. That figure was too optimistic. We withdrew it when the stricter evaluation replaced it. We keep both records here so that you can see how the first tests went.
Limits
- A model can rate a headline wrong. Outlets can be late or wrong.
- The prices come from public sources and can differ between venues. A brief error in a source can appear in a chart.
- The number of signals is small. A hit rate from a small sample says little. Look at the count next to each rate.
- Signals and simulated results are historical or simulated. They predict nothing. We model no fees and no slippage.
- If a source stops, the page shows the age of the data and a banner. It does not fill the gap.
General information, not investment advice or a recommendation. Automated ratings can be wrong. Signals and simulated results are historical or simulated and predict nothing.