Work

Through the early stages of the creator discovery launch, search quality was assessed anecdotally — a client complaint, a demo that went well — on top of basic volume and query data. The index held over six million creators, and whether the top twenty results for a query were the right twenty went unmeasured. At seed stage, with one product person, that was a deliberate trade-off.

After the acquisition, Creator Search was the primary creator-sourcing engine inside an enterprise platform, which meant its quality needed to be measurable: a golden set of queries with agreed-correct results, an accuracy metric the team could interrogate, and an observability program to catch regressions before clients did.

The first measurement put demographic model accuracy at 72%. That baseline turned an open question into a prioritized backlog, with every failure case in the golden set becoming an item of work. The program closed at 93% accuracy, with latency down 68%. The same period shipped a creator brand-safety score on the GARM framework, reading image, video, and caption to flag risk at the point of discovery.

The three tools came with three roadmaps and three owners, each with commitments already made to clients and leadership. The investment case argued two things: that the portfolio's growth targets did not work with the offerings sold separately, and that fixing this meant stopping some funded work to make room. The second argument consumed most of the alignment effort, because a priority reset means walking back commitments already made by other owners, which had to be negotiated stakeholder by stakeholder.

What shipped is one managed product: the 9.5M-member sampling community, ambassadors, and paid creators packaged, sold, and operated together. A large share of the work went into coordination, since the offering definition had to hold simultaneously across Sales (one thing to sell), Finance (one thing to price), and four engineering teams (one architecture to build against) without drifting every planning cycle.

The framework evaluated build-versus-buy against the roadmap rather than against the deal in front of us. Payments was not going to stay a one-off checkbox: sampling payouts, ambassador incentives, and affiliate commissions were all heading toward the same rail within eighteen months, and a vendor integration would have to be redone for each of them. Owning the rail meant building it once.

There was reasonable pushback. Finance preferred a specific vendor, Legal preferred the smallest possible compliance surface, and a credible vendor already existed for this exact use case, so taking on money-movement risk voluntarily required strong justification. The framework resolved it on a narrow point: the vendor route solved the immediate deal and left the roadmap use cases — payouts, incentives, commissions — each requiring its own re-integration.

The build scope covered admin-led managed transfers, full transaction history, and a new compliance path defined with Legal and FinOps. After launch it addressed 28% of previously lost Affable deals and reopened 24 stalled commercial conversations, and the platform now serves as the payment rail under Integrated Sampling.

The shift ran as a sequence of small changes rather than a program, with the case for each made separately. LLM tooling was argued for and secured. AI prototyping was embedded into the scoping process, so feasibility questions were answered before commitment instead of during the build. An agent prototype (on Mastra) was built ahead of engineering capacity, accepted as an unresourced bet so the team would have a working reference point when capacity opened up.

The clearest measurable evidence came from Intelligent Product Tagging, which was stuck around 10% model accuracy with no evaluation framework to explain why. Under the new operating model the team built a golden set, read failure cases against it, and treated each prompt revision as a hypothesis to test. Accuracy reached roughly 60% within a year across 25 enterprise pilot clients, with phased OKRs met along the way. That discipline is now the default for the team's AI work.

Cinemas closed in March 2020 and the company's core revenue went to zero with them, which compressed the discovery phase to near zero as well: the brief was a transactional streaming product on web, mobile, and TV, live in weeks. Alignment that would normally take a quarter happened in days, and the resulting plan went stale almost weekly as the market moved. Owning discovery, checkout, and cross-device in that environment meant re-deciding every week which of the previous week's calls still held.

It carried roughly 80% of company revenue, growing about 30% month over month, for as long as the theatres stayed shut. The second metric is support: voice volume came down 30%, through a reworked Help section and contextual chat, which scaled better than adding support capacity under that load.