The approach
Our member Infofarm took it upon themself to advise Materialise about architecture, design and implementation of a new Data Platform based in AWS cloud, using modern concepts like Lake House, aiming at putting in place a new foundation of enabling the IT department moving forward with its data journey.
Materialise
Materialise has over 30 years of experience in providing 3D printing services and software solutions for a variety of industries, including healthcare, automotive, aerospace, consumer goods, and art and design. Materialise is divided in three different divisions:
- Manufacturing: 3D printing solutions
- Software: software solutions for the 3D industry
- Medical : vertical specific, combining software and 3D printing
A clear ambition
The ambition of Materialise was to enable data driven working within their organization and use data as driving force in their decision-making process.
To realize this ambition, Materialise was looking for a partner to advise on Data Strategy and make sure the technology was well defined and put into motion. A project that fits Infofarm like a glove.
A future-proof foundation on AWS
Since Materialise decided to adopt a Cloud-first strategy for most of its new enterprise applications and solutions, and chose AWS as their main cloud provider, Infofarm was asked to bring all data to the cloud. We’ve set up a future-proof technical foundation with growth in mind, based on the AWS Well-Architected Framework and our own AWS reference architecture. With this architecture in mind we’ve setup a data platform using native AWS services only:
- Data Ingestion: Because Materialise has data coming from different source systems scattered around different departments within the company, it is also in need of different services to support the ingestion. For the technical foundation we have chosen to implement AWS Lambda and AWS Database Migration Service for batch processing, and Amazon Kinesis Data Streams for stream processing.
- Data Storage: At the center of the data platform we’ve set up a data lake on top of Amazon S3. We’ve chosen to create the traditional three-layered approach, where each layer is a separate S3 bucket with its own configuration and lifecycle policies.
- Data Processing: In the design of our reference architecture we’ve chosen for a serverless-first approach. This results in using AWS Glue Studio for creating the ETL jobs. To give the customer a no-code solution for cleaning and normalizing their data, we’ve also enabled the use of AWS Glue DataBrew.
- Data Consumption: Organizing the data and making it easily discoverable is one of the key features of the foundation. This enables us to use Amazon Athena as the query layer for our platform. We’ve made sure that all data within the data lake is easily accessible in a cost-efficient manner. Athena will be used to perform queries for ad-hoc questions or for data discovery, as well as a query connector between BI tools like Amazon QuickSight.
- Data Governance: Governing your data assets is a very important part of this foundation. AWS offers a variety of services that can help Materialise through their data governance journey:
- AWS Lake Formation for data security and centralized governance
- Amazon CloudWatch and Amazon CloudTrail for monitoring and audit trails
- AWS Glue Catalog and AWS Glue Crawlers for data discovery
AWS reference architecture used by Infofarm that combines all necessary AWS-services resulting in a qualitative Cloud Data Platform.
Since Materialise shifted to AWS-technology, Infofarm created a custom-made training for Materialise employees to make sure they get familiar with the services used in their new AWS Data Platform. The training covered the following topics: AWS Basics, Amazon S3, Amazon EC2, AWS Lambda Serverless functions, Logging & Monitoring, Data Platforms, Data Ingestion, Storage, Processing, Querying, Visualization, Governance & Machine Learning.
“We’ve been discussing the desire to work data-driven for long time, but we didn’t know where to start and how to do it. Infofarm helped to develop the necessary data strategy in a practical way. In six months, they created a working data platform. Within Materialise, the direction is clear, and we know what we must do to reach our ambition: moving forward to a more data-driven organization."
Jean-François Bauduin - Lead Enterprise Architect at Materialise
Results
After six months Materialise was left with a fully operational data platform on AWS. With this platform they now have a foundation, a basis, where they can start implementing their data workloads.
The platform was designed to support a cloud-first strategy. It allows Materialise not only to migrate their existing data workload, but also build new enterprise solutions in the cloud.
One of the key benefits of the data platform is the ability to support data coming from different source systems used from everywhere in the organization. The platform also provides a future-proof data lake making it easily accessible in a cost-efficient manner.
Another benefit of the platform is its serverless-first approach for data processing. This allows Materialise to really benefit of the reliability and scalability of the cloud without having to create and manage the infrastructure themselves.
Overall, this platform is a scalable and cost-effective solution for handling their data and data workloads. It supports the company’s cloud-first strategy and provides a future-proof technical foundation for growth. With its ability to ingest, store, process, and consume data, the platform enables Materialise to easily access and analyze their data for better decision-making.
About 
Infofarm is a Scale member and is part of our AWS Data & Analytics Competency. They are specialised in Data Strategy, Data Engineering and Data Analytics. Infofarm builds end-to-end data-driven solutions to bridge the gap between the Academical world of Computer Science and the IT-technical world of software engineering.