Architecture Overview

StringX is conceptually composed of two parts: Platform and CoreServices.

The platform provides all the horizontal services required to configure and run data transfer operations e.g: the UI, Server, Integration, job scheduling, logging, alerting, etc. and is structured as a set of microservices.

CoreServices are independent modules which uses intergration_config to operate data engineering accordingly. CoreServices are built in accordance with the StringX Specification, which describes the interface with which data can be moved between a source and a destination using StringX.

CoreServices also are packaged as Docker images, which allows total flexibility over the technologies used to implement them and easy deployment and deployment as well as holistically supporting by different environments.

A diagram of following picture is demonstrated how StringX is implemented: Architecture Overview

UI: Interface implemented for interacting with StringX API.
Server: Communicate with services using Rest API (future versions can have Asynchronous - Message Queue) for decoupling communication.
Integration: Onboard DataSource with required information about data source ( Connecion String, Secret Key, endpoint, etc.)
IntegrationConfig: Connection Database Objects with fully stored within StringX platform.
Core-Services: Function which provides services for interacting with data
- Transfer: Data will be transfered and migrated from source to target database, no matter what data type is.
- DataOps: Data quality evaluation will be performed on the onboarded data source, be able to integrate with CI trigger. or Data-CD trigger when perform data movement.
- Enablement: Conduct data visualization and semantic layer, publish visuals or dashboards, emmbed chart to your own application. As well as revert ELT to expose data out side StringX.
- Recommend: Run recommendation on data structure of data infrastructure.
Buffer: Temporary buffer storing data when Services need to spell data on a disk.