Scribd, Inc. is home to one of the world’s largest collections of human-created content.
Scribd’s products leverage one of the world’s largest collections of human-created content and intelligent tools to help people move from information access to real understanding and application.
This past year, Scribd used Gemini’s native PDF understanding and Gemini Enterprise batch prediction to run trust and safety classification across its entire user-generated content corpus of more than 400 million documents, spanning over 12 billion pages, in a matter of months.
Here were the results:
-
Classified 400M+ user-uploaded documents (12B+ pages of text and images) across Scribd and Slideshare
-
Completed the corpus-wide backfill in a matter of months, with Google Cloud scaling batch throughput to meet the timeline
-
Native PDF input meant more than 99% of the corpus was processed as-is, with no OCR, rendering, or screenshotting pipeline to build
-
Gemini Enterprise’s batch prediction at a 50% discount to interactive pricing made LLM classification viable at corpus scale
Trust and safety at the scale of an entire corpus
Scribd, Inc. is the parent company to four distinct products: Scribd, Slideshare, Everand, and Fable. Across Scribd and Slideshare, hundreds of millions of user-uploaded PDFs, presentations, and documents help people find information, build understanding, and finish projects. With that scale comes responsibility. We aim to balance access with protecting our communities. We leverage a mix of human and automated methods to review and best ensure the content on our platforms complies with our community rules. As the corpus continues to grow and technology evolves, this challenge requires even more resources.
Understanding a document requires reading its text and its images together, in context. Classification has to work across all possible use cases, all possible languages, all possible contexts. There is no single solution that can translate cleanly across all of it. And each policy area traditionally demanded its own specialized detection model, which meant either years of in-house engineering effort or specialized vendor solutions that don’t fit the economics of a 400-million-document backfill. The team evaluated several off-the-shelf moderation tools and open models, but none delivered the quality they needed at their scale.
“This is a genuinely hard problem that we have been working on for a long time. Every category of content behaves differently, and historically each one required its own specialized solution. Gemini collapsed all of that into one model, one prompt, and one pipeline.” – Sachin Sebastian, Senior Engineering Manager, Scribd, Inc.
Why Gemini: PDFs are a first-class input
The turning point was realizing that Gemini treats Scribd’s corpus the way it actually exists: as PDFs. Gemini accepts PDF input natively and reads each page as both text and image, so a single multimodal model could evaluate everything from dense text documents to image-heavy presentations, with no OCR pipeline, page rendering, or screenshot infrastructure in between. Because Gemini processes each PDF page at a fixed, predictable token count, costs scale linearly and stay low even across 12 billion pages.
After benchmarking model families and versions, the team selected Gemini 2.5 Flash Lite as the classification workhorse, with Gemini 2.5 Pro serving as an LLM judge in a full second consistency pass over the corpus to validate output quality. In the team’s evaluations, Gemini’s multimodal understanding caught visual policy signals that text-only moderation endpoints routinely missed.
“Gemini’s peculiar advantage is that it meets our content in its native format. It reads the text, layout, and images of a PDF directly. More than 99% of our corpus went in exactly as it lives on our site without any pre-processing” – Sachin Sebastian, Senior Engineering Manager, Scribd, Inc.
Batch prediction, simple enough to bet the corpus on
The execution model was deliberately simple. Documents were staged in Cloud Storage, submitted to Gemini Enterprise batch prediction, and the results flowed back into the team’s data platform for downstream analysis. There was no serving infrastructure to operate, no rate-limiting logic to write, and no GPU capacity to manage.
Batch pricing, at 50% below interactive rates, is what made the economics work at corpus scale. The team later layered on Gemini Enterprise’s implicit prefix caching, restructuring prompts so the static policy text hit the cache, which pushed efficiency further with no loss in classification quality.
A partnership measured in throughput
Processing 400 million documents is ultimately a throughput problem, and this is where the partnership with Google Cloud mattered most. Scribd’s team connected directly with Google Cloud engineering and product to plan the backfill, advise on region strategy, and make sure the right capacity was in place ahead of launch.
As the backfill ramped up, Google Cloud worked closely with the team to scale throughput to the demands of the project. The effect was dramatic: batch jobs began completing far faster than projected, and for much of the run Gemini Enterprise was not the bottleneck. Scribd’s own upstream pipeline was.
“Google Cloud didn’t just answer support tickets. They partnered with us on the backfill, and there were stretches where Gemini Enterprise finished work faster than our own systems could produce it. That is a good problem to have.” – Sachin Sebastian, Senior Engineering Manager, Scribd, Inc.
What’s next
The backfill is now the foundation of an ongoing program: newly uploaded content flows through the same Gemini classification pipeline, keeping the corpus continuously evaluated rather than periodically cleaned. And because the pattern of PDFs in Cloud Storage, Gemini batch prediction, and results in the lakehouse proved so operationally simple, the team is applying it to a growing set of content-understanding workloads across its platforms.
“This project changed how we think about our roadmap. Work we had classified as multi-year, multi-team efforts is now a prompt, a batch pipeline, and a few weeks of runtime.” – Sachin Sebastian, Senior Engineering Manager, Scribd, Inc.
This work was a collaboration between Google Cloud and Scribd. We’d like to thank everyone involved for their support throughout this project:
- Scribd Engineering: Anish Kumar, Jeanie Lam, James Watkins, Hima Alladi
- Scribd Applied Research: Rafael Pedrosa Lacerda de Melo, Kara Killough, Eric Chang
- Scribd Product: Seyoon Kim, Nicole Pauls
- Google Cloud AI Batch Inference team: James Liu, Digvijay Singh, Wei-chung Wang, Yan Wang, Kun Shi
- Google Cloud Customer Engineer: Jennifer Liang
