QiH is an online digital marketing company which runs marketing activities across a number of different channels.
About the project
QiH is an online digital marketing company working across a number of different channels with its own affiliate networks, Tau Affiliate and Offer Tonic. Its goal is to attract new customers to sites and keep existing customers engaged. Founded in 2014, the company now works with a portfolio of more than 50 sites including secretslots.com and aims to launch a new brand or site once every quarter.
The company draws on data from multiple sources, such as social media platforms, keyword advertising and player activity streams, to devise campaigns and evaluate their success. This data is used to provide daily KPI snapshots for the whole company, together with detailed reports for each “business owner” for the specific brands they manage and earnings reports for affiliates.
Over time, their in-house system became unreliable, slow, expensive to run and difficult to manage and maintain. Ancoris has re-developed the system to create a reporting platform that gives business users faster access to accurate information, while cutting software and processing costs and shortening the time needed to onboard new brands or add data from new marketing channels. The new platform also provides a robust foundation to apply new technologies such as machine learning to improve its decision making. With Ancoris providing the solution as a managed service, QiH can concentrate on exploiting data, not managing it.
Then
- Unreliable data ingestion. At least one data pipeline every month was either failing entirely or loading only partial data. Fixing issues through the third-party data pipeline tool used to create these connections required greater technical expertise than was available in house.
- Rapidly rising costs. The charges for BigQuery, which uses consumption-based pricing, were growing at a much faster rate than the business, because all the data held in the system was being queried every time any report was accessed. In addition, a premium of up to 50% was charged regularly for exceeding the contracted monthly allowance for the third party data pipeline tool
- Poor experience for users. Reports were taking up to ten minutes to run and load. Because data ingestion wasn’t reliable, users didn’t have full confidence in the reports and were wasting time drilling down into the underlying data to manually check the results.
- High management and maintenance overhead. Adding a new brand or marketing channel was time-consuming and difficult because it involved combing through lines of code to make changes to “hard coded” SQL statements. Maintaining the system was hampered by inconsistent naming conventions, poor data organisation and a lack of documentation for dataflows and dependencies.
-2.jpeg)
Now
- Robust data ingestion and transformation, created using third-party tools Fivetran and dbt.
- Significant reduction in costs by: moving to Fivetran for data ingestion; partitioning the data in BigQuery; and improving how data is organised, to minimise the volume of data processed when reports are viewed.
- Fast, accurate reporting for users, with reports generated in seconds. Users now have greater trust that what they’re seeing in reports is accurate and can be used immediately to make decisions.
- Management and maintenance woes are a thing of the past. New data sources can be added in just a couple of hours, with no need for specialist coding or data engineering skills. Dataflows and dependencies are documented and charted automatically in dbt.
- Solid foundation for applying modern data analytics techniques like machine learning to support tasks such as predicting expected customer lifetime value.