Page 1 of 1

Data Services & Micro-Service Architecture use-cases

Posted: Mon May 13, 2019 10:39 am
by JonathonO
Is Data Services suitable for micro-service arch. use cases, specifically for reporting on ETL flows?

The main reason for my question is that I was thinking about an endpoint for sending notifications out to various recipients and the other for logging job stats e.g. cpu, memory usage and records processed. The payloads would - for now - only require the default data that is returned from ExecuteGraph/Jobflow components; no custom fields.

These services would not need to run sequentially/synchronously as part of the ETL flow; they only need to be invoked by the flow. I'm also conscious of the design pattern regarding Jobflows and Subgraphs with token tracking so I'd like to avoid wrapping these services into subgraphs and executing them in the ETL flow.

The API calls wouldn't post too large a payload. But we'd be looking at 10000s of calls per day.