Reduce Update Latency and Build a Robust Pipeline Architecture for Ingesting at Scale

These days it is not uncommon for search engines to ingest billions of data updates per day. As our data scales, it becomes challenging to keep update delays in Solr to the minimum at such volume without affecting search accuracy. Wayfair came up with an architecture that cut down update latency from hours to seconds. They are replacing traditional ETL pipeline by building a new data ingestion platform for Solr based on an event-driven streaming architecture. Learn how the data pipelines built on this new platform are helping us push updates to SolrCloud much faster than ever. This new platform has fundamentally changed the way we looked at data ingestion challenge in Solr. However, it did not happen without its own challenges. Hear best practices, lessons learned from the experience of building this platform and the practical aspects of implementing it.



Suyash Sonawane, Staff Software Engineer, Wayfair

activate19,Apache Solr,search,solr developer,solrcloud,Wayfair,