Biginsight X AWS

2022년 09월 01일고객 사례

Amazon Web Service (AWS) provides a massive global public cloud infrastructure that allows you to quickly innovate, experiment and iterate your products and services. Instead of waiting weeks or months, you can instantly deploy new applications, scale up as your workload grows, and scale down based on demand.

Get the most out of Amazon Web Services

Modern data architecture of Biginsight using AWS

Raw layer (AWS S3)

It is a layer that can convert data into the desired format according to future requirements by maintaining the initial appearance of data as much as possible from the perspective of security and data reprocessing. The storage used in the raw layer should be able to store large amounts of data for a long period of time at a low price.

Standardized layer (AWS S3)

The standardized layer is usually integrated with the raw layer. The purpose of the Standardized layer is to change the data (e.g. avro, parquet, json, tsv, and csvetc) to a file format that is suitable for big data so that the data can be used immediately. With the suitable file format, it enables schema validation, schema evolution, data quality control, and data cleaning, and allows partitioning with column format that is already available for analysis queries. After the standardized layer, all data is changed to structured, which is to efficiently operate the data schema using like AWS Athena, Prestoetc. With this tool, you will only need to look at one place when there is any data schema invalidation.

Conformed layer (Clickhouse, Vitess.tenant, Vitess.tenant_application)

Typically, there is a domain model that is commonly used by any company, and those entities are called conformed entities. Conformed entities are usually the company's main business model, they are centrally managed in the conformed layer hence they have to be developed as early as possible.

Characteristics of Modern Data Architecture with AWS