Detecting Free-Tier API Abuse Without Blocking Your AI Agent Traffic

A support-style report came in that read like nothing: "might be some free usage leak." No error, no failed charges, no red flag in the dashboard. Every metric that's supposed to catch abuse looked completely normal: charge attempts, charge acceptances, revenue per run. That's usually where this kind of investigation stalls. It's also exactly where it shouldn't.

TL;DR

• A scripted free-tier farm can look identical to healthy usage in billing metrics, because on pay-per-event platforms, free-tier charges are often funded by platform credit, not real revenue.
• The signal that actually catches it isn't billing. It's a shared network fingerprint across accounts that should have no reason to share one.
• Blocking every programmatic channel at once is the easy fix and the wrong one. Check each channel's own behavior before you block it, or you'll cut off legitimate traffic along with the farm.
• The durable fix is a per-account daily quota that's origin-agnostic and two-tier: one counter that refunds on a real charge, one that never does.

Why "the charges are landing" doesn't rule out abuse

The first instinct when a usage-leak report comes in is to check whether billing is actually working. Are charge attempts succeeding? Is revenue accruing? Is anything failing silently? On our platform (we run a fleet of pay-per-event automation tools on Apify), every one of those checks came back clean. Charge attempts matched charge acceptances one-to-one. Revenue was accruing normally for the tier in question.

That's the trap. On a marketplace platform with pay-per-event pricing, a free-tier account's usage is frequently backed by platform-issued credit. The developer still gets paid for it, up to a cap, funded by the platform rather than the end user. A scripted account riding that credit charges just as cleanly as a real trial user. A clean charge-acceptance rate tells you your billing code works. It tells you nothing about whether the traffic behind it is legitimate.

That distinction matters beyond this one platform. Any system where a "free" tier is funded by someone other than the user generating the usage, whether that's trial credits, sponsor-funded quotas, or promotional API keys, has the same blind spot. Billing health and traffic legitimacy are different questions, and it's easy to only ever ask the first one.

The signal that actually reveals a farm

Once billing was ruled out as an explanation, the next step was to stop asking whether the traffic was being charged correctly and start asking whether it behaved like a person. A few queries against run-level telemetry (timestamp, tier, origin, a hashed account identifier, and a hashed session or proxy fingerprint) turned up a pattern billing metrics never would have surfaced.

One shared session fingerprint touched five to eight distinct account identities, every single day, and a fresh fingerprint with a fresh cohort of accounts showed up the next day. That's not two users on the same coffee-shop Wi-Fi. That's infrastructure being rotated on a schedule, alongside the accounts running through it.

Three more signals confirmed it, layered on top of the fingerprint sharing:

• Tight, mechanical timing. The same account firing a request every 30–90 seconds for hours. No human browses like that.
• Escalating volume, flat headcount. Daily run volume climbed roughly sixfold over a week while the number of distinct accounts involved barely moved. That's a farm scaling throughput per identity, not organic signups adding new users.
• Zero conversion. None of the flagged accounts had ever generated a single paid-tier run, anywhere in the platform's history.

Any one of these alone could have an innocent explanation. A power user might hit the API every minute. A single busy account might ramp up its own volume for a legitimate reason. A shared fingerprint across accounts that have no business sharing one, combined with mechanical timing and zero paid conversion, stopped being explainable as anything other than a script.

The mistake we almost made

Once a farm is confirmed on one traffic channel, the obvious fix is to lock that channel down for free-tier accounts entirely, and that's exactly what shipped first. Our platform has more than one way for software, rather than a human clicking through a UI, to trigger a run: direct API calls, and a newer integration layer that lets AI agents call the same tools. The confirmed farm came in through direct API calls. Since the fix needed to close off programmatic access broadly, the AI-agent channel got blocked in the same pass, on the reasoning that a determined abuser could just pivot from one programmatic entry point to the other.

That reasoning isn't unfounded, but it hadn't been checked against actual data for the AI-agent channel specifically. It turned out to matter for a reason that had nothing to do with farms: that channel had been growing fast and organically, as more AI agents discovered and started calling the tools directly. Blocking it wasn't just a false positive on a security check. It was cutting off a real, fast-growing acquisition channel to plug a hole that channel didn't have.

Re-running the same shared-fingerprint analysis, scoped specifically to AI-agent traffic, told a completely different story from the direct-API channel: no shared fingerprints across accounts, no mechanical timing, and a long tail of distinct callers each running the tool once or twice over several weeks. That's what organic discovery looks like, not a farm.

There was a second, quieter mistake in the middle of re-checking this. An earlier query, a simple "top accounts by usage" ranking, hadn't filtered by traffic channel, so it mixed direct-API and AI-agent accounts into one list. A few accounts that only ever showed up via the AI-agent channel got misread as overlapping with the confirmed API-channel farm, purely because they appeared in the same unfiltered ranking. Only re-running the specific claim, checking whether those exact account IDs appeared under both channels against properly filtered data, caught the error.

An unfiltered ranking that happens to contain the accounts you're worried about is not the same evidence as a filtered query that proves the overlap. It's a small distinction, and it's exactly the kind of thing that's easy to skip when a hypothesis already feels confirmed.

The fix that generalizes

The corrected fix has two parts, and only one of them cares which channel the traffic came in on.

Block the channel with zero legitimate value, and only that one. Direct API access for free-tier accounts got blocked before any billable work happens. Free access through the interactive console, and the AI-agent channel, stayed open, because the data showed no abuse on either.

Add a per-account daily quota that doesn't care about the channel at all. This is the part that actually generalizes, and it's worth stealing for any multi-tenant, pay-per-event system. Instead of enumerating every programmatic entry point and blocking them one at a time, a losing game since a scripted account can always find a channel that hasn't been locked down yet, cap how many free runs a single account identity gets per day, regardless of how it got in.

The quota is deliberately two-tier, because a single counter has a specific failure mode. The obvious design, a counter that refunds itself whenever a charge is accepted on the theory that an accepted charge proves the run was worth it, has the exact same blind spot as trusting the billing metrics in the first place. A free-tier charge accepted against platform-funded credit isn't evidence the run made anyone money. It just means the platform paid instead of the customer.

So the quota tracks two numbers per account per day: one refunded when a charge genuinely lands, which catches accounts that flat-out can't be billed, and a second, harder ceiling that's never refunded, which catches accounts that get billed successfully every time but never once convert to a paying plan. Genuine trial users have plenty of headroom under both. A farm, automated or not, runs into the second one fast.

The whole mechanism fails open by design. If the quota-tracking service is unreachable, every check defaults to allowed. A dependency outage should never be the reason a paying customer's run gets blocked.

What we'd tell another team building on a platform like this

A few things generalized past this one incident, worth carrying into any system where "free" usage is funded by something other than the end user's wallet.

Don't trust a healthy charge-acceptance rate as proof of legitimate traffic if there's any world where free usage is funded by someone other than the user generating it. Check the shape of the traffic, not just whether billing accepted it.

A shared network fingerprint across accounts is a stronger signal than volume alone. A single very active account might just be an enthusiastic user. Multiple accounts sharing one fingerprint have no innocent explanation.

Don't generalize a fix across channels you haven't actually checked, even when the reasoning sounds airtight. "They could pivot there too" is a real risk worth defending against, but defend against it with a mechanism that doesn't care which channel the abuse uses, like a channel-agnostic quota, instead of pre-emptively blocking a channel you haven't confirmed is a problem.

Re-verify any claim built on an unfiltered query before it drives a decision. "These accounts showed up in the same list" is a much weaker claim than "these accounts overlap under this specific filter." The difference is one extra query, and it's cheap insurance against shipping a fix based on a coincidence.

None of this required new infrastructure. It required looking at the traffic pattern instead of the billing outcome, and checking a channel's own data before writing it off.