OptiFlowArchitecture
OptiFlow employs a structured approach to defined workflows, organizing them within a container called a Workflow Group. Each Workflow Group is essentially a containerized .NET console application, enriched with specific data annotations that enable OptiFlow to seamlessly identify its contents.
Utilizing a reflective and metadata gathering discovery process, OptiFlow adeptly identifies the entry point for a Workflow Group, along with the Assembly and Type names for each individual Workflow within that group.
Post-discovery, OptiFlow facilitates the scheduling of workflows for recurring execution based on their predefined schedules. The OptiFlow server efficiently manages a schedule of recurring workflows and executes them in an isolated and distributed manner through Kubernetes Jobs. This approach ensures a seamless and scalable execution environment for recurring tasks.
Description of Flow
Workflow Group Discovery
Container images are registered with OptiFlow and scheduled for Discovery Execution. In Discovery mode, OptiFlow executes the container to identify crucial elements such as the Workflow Group, individual Workflows, Workflow Operations, and other associated metadata.
Supplementary metadata, including Workflow Tags, Operation Data Types, and the option to halt execution upon encountering an error, is also captured during the Discovery process.
Following the completion of Discovery, OptiFlow reports back to the server. Users can then review the contents of the Workflow Group and, as needed, create a scheduled Workflow Group based on their preferences. This streamlined process ensures transparency and user control over the configuration and scheduling of workflows within OptiFlow.
Discovered Workflow Review and Scheduling
Following the Discovery process, users gain the capability to visually identify the Workflows and Operations uncovered. This empowers them to seamlessly create a Scheduled Workflow Group based on the discovered elements.
Individual Workflow View
Viewing a Diff of Changes in the Workflow
Scheduled Workflow Execution
Once a Workflow Group is marked for recurring execution, OptiFlow initiates the creation of a recurring Quartz.NET trigger, executing each Workflow based on its defined schedule. The execution process mirrors the Discovery phase, with OptiFlow now running the container to execute the Workflow.
Throughout each Workflow execution, the OptiFlow library diligently reports each Operation Result back to the OptiFlow server. This reporting mechanism serves to track the Workflow lifecycle and generate insightful Workflow execution metrics.
Within each Workflow execution, individual Workflow Operations can opt to save or retrieve arbitrary states from the State Store. Additionally, they have the flexibility to store sampling reports in OptiFlow's Report Store. The management of these stores is seamlessly handled through the OptiFlow library, ensuring storage in a remote location for future reference.
Workflow metrics encompass Success and Error rates at both the Workflow and Operation levels, along with average duration metrics. This comprehensive reporting mechanism provides valuable insights into the performance and outcomes of each Workflow and its constituent Operations.
Workflow Changes
To uphold a singular source of truth for Workflows, any modifications must be made directly within the source code of the Workflow. Following adjustments, the Workflow can then be rescheduled with OptiFlow after undergoing a Discovery process and a review of the updated Workflow Group container. This ensures that the source code remains the authoritative reference for Workflow configurations, promoting consistency and accuracy.