DataLake and Analytics for Digital Exam Platform on AWS
We were able to unify multi source(including on-premise) data sets, created future proof analytical platform for delivering a hierarchical student performance reports.
About the client
Singapore based start-up focusing on digitizing the exam process in educational institutes.
Business challenges
- An Ed tech company had a challenge of building a scalable, cost effective, in-platform Student Performance Analytics
- They were looking for a future proof, Smart Digital Hub for their Analytics within 8 weeks
Project Scope
- Design & Architect DataLake on AWS
- Hydration of DataLake from multiple sources
- Student Performance analysis reports in Quick sight
- Data Sources : MySQL
Our approach
Understand
Define
Design
Transform
Reporting Dashboards
01. Understand
- In a series of collaborations with the customer teams, Identified volume and sources of data
02. Define
- Worked closely with client’s business teams and outlined the reports that would help them make meaningful decisions
03. Design
- Mapped the data to the end reports
- Architected leveraging Nimbus Insights and created pipeline for data to flow seamlessly from various data sources including CDC to AWS Quick sight
04. Transform
- Raw data was cleansed and stored in S3 as compressed parquet files
- Used custom Python scripts for Data Ingestion from data sources into the DataLake with Data partitioning
- Created data schema in Athena for ingested data to flow into Quick sight
05. Reporting Dashboards
- Provided custom dashboards with meaningful insights on Exam Statistics, Score Analysis, Question Paper Analysis, which helped customer to continuously improve their digital exam platform
Outcomes
Nimbus Insights Smart Digital Hub was used to build a scalable, cost effective, DataLake
By leveraging Smart Digital Hub, we were able to deliver within a short span of 6 weeks
Quick sight Embed was used to deliver a in-platform experience for the reports with drill downs to the student level of each university
Other AWS services used were DataLake, S3, Glue, Athena and DynamoDB
Write to us at contactus@1cloudhub.com if you have any queries.
