We were processing 2 million API calls per day when our billing system crashed. Not because of traffic, because we couldn't figure out how to count usage correctly.
The problem wasn't our AI model or our API infrastructure. It was ingestion, the simple act of getting usage data from our application into our billing system.
The Ingestion Problem Nobody Talks About
Every SaaS company faces the same challenge: how do you reliably capture every API call, every user action, every resource consumption event, and turn it into billable usage?
Sounds simple. Until you try it at scale.
Our first attempt was basic logging. Every API call wrote a log entry: user ID, timestamp, endpoint, tokens consumed. We processed these logs nightly to generate usage reports. Worked great for 1,000 calls per day. Broke completely at 100,000 calls.
The issues piled up fast:
- Log files got corrupted when servers restarted
- Daily processing took 6 hours and frequently failed
- Customers couldn't see real-time usage
- Missing log entries meant lost revenue
- Duplicate entries meant overcharging customers
Why Ingestion Is Harder Than It Looks
Usage ingestion isn't just data collection, it's a distributed systems problem. Here's what makes it complex:
Volume and Velocity
Modern applications generate thousands of usage events per second. Traditional databases can't handle the write volume. Batch processing creates delays. Customers expect to see their usage immediately, not tomorrow.
Reliability Requirements
Every missed event is lost revenue. Every duplicate event is an angry customer. Your ingestion system needs to be more reliable than your application itself, because billing mistakes are existential threats to your business.
Data Quality Issues
Usage events arrive out of order. Timestamps are wrong. Customer IDs are malformed. API keys are expired. Your ingestion system needs to validate, clean, and normalize data in real-time while maintaining performance.
Multi-Tenant Complexity
In a multi-tenant system, you need to ensure customer A's usage never gets attributed to customer B. This requires careful data isolation, tenant-aware processing, and secure API design. Learn about multi-tenant security best practices.
The DIY Ingestion Trap
Like most engineering teams, we thought we could build ingestion ourselves. Here's what we learned:
Attempt 1: Database Writes
Direct database writes from our API servers. Simple, fast, reliable, until database connections became our bottleneck. We scaled horizontally, added connection pooling, implemented circuit breakers. Each fix added complexity while reducing reliability.
Attempt 2: Message Queues
We moved to RabbitMQ, then Kafka. Better, but now we had to manage message queue infrastructure. Message loss, consumer failures, partition rebalancing, our ingestion reliability depended on systems we didn't fully understand.
Attempt 3: Event Streaming
We built custom event streaming with Redis, then tried Apache Pulsar. Each technology solved some problems while creating new ones. Our ingestion system became more complex than our core product.
After 14 months and three complete rewrites, we had a fragile ingestion system that required constant maintenance and still missed 0.1% of events. That doesn't sound like much until you realize it's thousands of dollars in lost revenue every month.
What Professional Ingestion Looks Like
When we migrated to UsageBox, we discovered what purpose-built ingestion actually means:
Serverless Scalability
UsageBox's ingestion endpoints scale automatically. No capacity planning, no infrastructure management, no performance tuning. From 100 events per day to 10 million events per day without changing anything. See how serverless scaling reduces costs.
Real-Time Processing
Usage events are validated, rated, and stored in real-time. Customers see their usage updating live. No more daily batch jobs or delayed reporting. Usage data flows through the system in milliseconds, not hours.
Built-In Reliability
Automatic retries, duplicate detection, data validation, and error handling are built-in. Events are processed exactly once, even during network failures or system outages. We stopped losing revenue to ingestion problems.
Multi-Tenant by Design
Every ingestion request includes tenant context, API key validation, and security checks. Customer data stays isolated automatically. We never worry about cross-tenant data leaks or billing errors.
Streaming ingestion pipeline
& SDKs] --> Edge[Authenticated API Edge] Edge --> Validator[Schema &
Quota Validation] Validator --> Queue[(Durable Queue
Pub/Sub or Kafka)] Queue --> Processor[Rating &
Deduplication Workers] Processor --> Ledger[(Billing Ledger
Durable Store)] Processor --> Metrics[(Usage Metrics
Aggregations)] Ledger --> Billing[Billing Engine
& Invoices] Metrics --> Dashboard[Customer Dashboards
& Alerts] Billing --> Finance[Finance Systems] Dashboard --> Customers[Customer Insights]
The Technical Architecture That Actually Works
UsageBox's ingestion architecture solved problems we didn't even know we had:
Cloud Run Processing
Ingestion endpoints run on Cloud Run, scaling from zero to thousands of instances automatically. We pay per request, not per server. During traffic spikes, the system handles the load without manual intervention.
Firestore Storage
Usage events are stored in Firestore with automatic replication and backup. Data is available for querying instantly, with built-in indexes for fast retrieval. No database administration required.
Firebase Identity Integration
Authentication integrates with Firebase Identity, so we don't need to manage API keys or user credentials. Customers use their existing authentication tokens, reducing integration complexity. Understand Firebase Identity integration.
Deterministic Latency
Ingestion requests complete in under 100ms, even at scale. Our API performance doesn't degrade when we add usage tracking. Customers get fast responses while we capture comprehensive usage data.
The Business Impact of Proper Ingestion
Reliable ingestion transformed our business in ways we didn't expect:
Revenue Recovery
We discovered we were under-billing by 12% due to ingestion gaps. Professional ingestion recovered thousands in monthly revenue we didn't know we were losing.
Customer Trust
Real-time usage visibility improved customer satisfaction dramatically. When customers can see their consumption patterns, they trust the billing process. Support tickets about usage discrepancies dropped by 80%.
Operational Simplicity
Our engineering team stopped maintaining ingestion infrastructure and focused on our core product. We went from 3 engineers working on billing infrastructure to zero, while improving reliability and performance.
Enterprise Sales
Professional ingestion capabilities accelerated enterprise deals. Large customers expect detailed usage tracking, audit trails, and real-time reporting. We could provide these capabilities immediately instead of building them custom.
Key Lessons About Usage Ingestion
We learned that ingestion is a specialized problem that requires specialized solutions:
Don't Build What You Can Buy
Usage ingestion looks simple but becomes complex at scale. The problems, reliability, performance, multi-tenancy, real-time processing, are well-understood by billing platforms. Building your own solution means solving solved problems.
Reliability Is Everything
Ingestion failures directly impact revenue. Every missed event is lost money. Every duplicate event is customer trust lost. Professional ingestion platforms are designed for financial-grade reliability.
Real-Time Matters
Customers expect to see their usage immediately. Batch processing creates poor customer experience and billing surprises. Real-time ingestion enables transparent, predictable billing.
Scale Changes Everything
Solutions that work at 1,000 events per day break at 1 million events per day. Design for scale from the beginning, or plan to rewrite your ingestion system multiple times.
Making the Switch to Professional Ingestion
Migrating to UsageBox's ingestion system was surprisingly straightforward:
Step 1: Replace our custom API calls with UsageBox ingestion endpoints
Step 2: Configure authentication and tenant isolation
Step 3: Update our dashboard to use UsageBox's real-time APIs
Step 4: Decommission our custom ingestion infrastructure
The migration took two weeks and immediately improved reliability, performance, and customer experience. We went from 99.9% ingestion reliability to 99.99%+ while reducing infrastructure costs.
The Bottom Line on Usage Ingestion
Usage ingestion is the foundation of usage-based billing. If you can't reliably capture and process usage events, nothing else matters, not your pricing models, not your customer dashboards, not your revenue recognition.
Professional ingestion platforms like UsageBox solve problems that every usage-based business faces: scale, reliability, real-time processing, and multi-tenant isolation. They've already solved the complex distributed systems problems so you can focus on your core product.
Two years later, we process millions of usage events daily without thinking about ingestion. Our events flow reliably from application to billing system in real-time. Customers see their usage instantly. We capture every billable event. We stopped building billing infrastructure and started building better AI products.
Sometimes the best technical decision is recognizing that ingestion infrastructure requires specialized expertise. Let the experts handle event streaming, distributed systems, and financial-grade reliability while you focus on what makes your product special.
Reliable ingestion isn't just about capturing usage data, it's about building trust with customers, ensuring revenue accuracy, and enabling your team to focus on innovation instead of infrastructure maintenance.