GitOps Guide: ArgoCD vs Flux

GitOps has become an extremely popular way to manage Kubernetes infrastructure and applications. By using Git as the single source of truth, GitOps enables infrastructure-as-code practices and automates deployment of applications to Kubernetes. This approach is being adopted by many companies nowadays, so I wanted to share our GitOps journey with the series of posts on this topic.

Let's start with comparing two main players in this field - ArgoCD and Flux (v2).

Core Functionality

ArgoCD and Flux have substantial overlap in their fundamental GitOps capabilities as they comply with OpenGitOps standards. Both tools reconcile application state in Kubernetes clusters based on declarations sourced from Git repositories. They support common methods of templating manifests like Helm and Kustomize out-of-the-box.

Additional baseline functionality provided includes:

• Auto-syncing to continuously apply desired state from Git
• Pruning on deletions
• Extensive CLI functionality to manage applications imperatively and declaratively
• Multi-tenancy to support different teams and tenants

At their core, both achieve the primary workflow of maintaining infrastructure-as-code and auto-deploying application specifications stored in version control.

They take diverging implementation paths in areas like UI, multi-tenancy, sync flexibility, etc. But fundamentally, either tool can provide automated GitOps pipelines with guardrails.

User Interface

One major difference between ArgoCD and Flux is the UI. ArgoCD ships with a built-in web-based UI that provides management capabilities and visibility into applications and clusters. The ArgoCD UI enables tasks like:

• Visualizing applications and their status across multiple Kubernetes clusters
• Viewing and comparing differences between live and desired application state
• Initiating syncs to fix any drift (full or partial)
• Browsing configurations like repositories, projects, and settings

In contrast, Flux does not include a UI component in the core project. The assumption is that users will utilize the flux CLI or integrate with an external UI layer. For example, Weaveworks offers the Weave GitOps UI as a supported add-on to Flux for achieving things like visualizing deployments across environments. Since Flux is focused on programmability, the UI is a separate concern.

Multi-tenancy

ArgoCD builds in capabilities for managing multiple teams and environments within a single control plane. With first-class abstraction like AppProjects it allows administrators to logically group applications and their resources. Fine-grained RBAC can then be applied to restrict what users can access or modify. It comes at a cost of maintaining "yet another rbac" outside of native k8s.

SSO integration and identity services in ArgoCD also enable centrally managing authentication and mapping identities to policies. Along with projects, this allows securing GitOps workflows across many clusters and applications.

apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: my-project
  namespace: argocd
spec:
  roles:
  # A role which provides read-only access to all applications in the project
  - name: read-only
    description: Read-only privileges to my-project
    policies:
    - p, proj:my-project:read-only, applications, get, my-project/*, allow
    groups:
    - my-oidc-group

Flux defers to Kubernetes’ native RBAC to control the multi-tenanancy, by impersonating (assuming the identity of) a service account mentioned in the Flux apply specifications. ArgoCD actually lacks this feature to enable a nice layer of isolation and there is long lasting discussion on adding it to the project.

In Flux authorization is constrained to the permissions granted to the specified service account. This allows users to tune access controls to the Kubernetes objects their Flux controllers act upon.

apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: testrelease
  namespace: apps
spec:
  serviceAccountName: dev-team
  ...

So for a centralized control plane with many applications and users, ArgoCD simplifies applying security policies. Teams that require intricate permissions management can use Flux's impersonation approach to customize access.

Resources Sync

A core function of any GitOps tool is to automatically reconcile differences between the desired state in Git and the live state of a cluster. This "syncing" ensures environments conform to the specifications in Git.

ArgoCD provides flexible options to control how and when syncs take place. For example, the CLI and UI allow triggering selective syncs to only apply changes to specific resources rather than the whole application. This finer granularity speeds up development cycles.

Partial syncs can also run based on the sync status - out-of-sync vs in-sync resources. Along with comparing live vs desired state before syncing, ArgoCD offers precise control over the reconciliation process.

Additionally, ArgoCD has the concept of SyncWindows which act as scheduled maintenance windows for when auto-syncs are permitted. This ensures changes happen within an allowed timeframe rather than unexpectedly.

SyncWaves allow further customizing the order of operations when applying manifests during a sync, which is useful for complex dependencies. And for advanced use cases, ArgoCD supports sync phases/hooks to execute scripts before/after syncs.

In contrast, Flux takes a more declarative approach - the system continuously attempts to reach desired state without finer-grained sync handling. The main options are to suspend or enable reconciliation through the controllers.

The sync functionality plays into the differing philosophies - ArgoCD exposing more levers vs Flux being more hands-off. Depending on the scale of an implementation, one may be better suited than the other.

Dealing With State Drift

Continuing on the previous item, ArgoCD and Flux has a different approach to state "drift" (when the live system diverges from the declared state)

As mentioned above, ArgoCD provides multiple ways to surface and correct drift:

• The CLI and UI allow comparing applications to spot differences between live vs desired state, making drift easy to catch visually.
• Automatically syncing (with SelfHeal option) fixes any drift, ensuring components get regularly reconciled to Git truth.
• For Helm-based applications, ArgoCD's helm template approach enables identifying changes in rendered manifests that could cause drift from Git.

❯ argocd app diff --revision feat/updrecontime argocd/k8s-argocd

===== /ConfigMap argocd/argocd-cm ======
58c58
<   timeout.reconciliation: 180s
---
>   timeout.reconciliation: 45s

Drift detection and reconciliation is considered experimental by Flux and has to be enabled using feature gates. So when keeping the default Flux configuration you can end up in situation where the live state deviated from the declared state in Git.

Ecosystem and Extensibility

The Argo Project has a rich ecosystem, including widely adopted projects like Argo Workflows and Argo Rollouts, which also integrate cleanly with ArgoCD since they are all hosted under the the same umbrella. This provides tested interoperability out-of-the-box.

In addition, ArgoCD's config management plugins (CMPs) allow extending support for various manifest generation tools like Kustomize, Helm, or Helmfile. CMPs can chain together configurations into a pipeline producing Kubernetes YAML - enabling flexibility to fit any possible needs of platform administrators or tenants.

On the other hand, Flux v2 emphasizes customizability following the UNIX philosophy of "do one thing well" - hence targeting specific controllers for CD functionality.

Scalability

Both ArgoCD and Flux scale well for production-grade use cases. I haven't had a chance to play around with Flux at scale, but it employs a pull-based Kubernetes operator pattern which relies on watches and caches to maintain efficiency. The controllers horizontally shard allowing Flux to deal with high number of applications.

As for ArgoCD, we'll leave this topic for future article as there is a lot of lessons we have to share, while we adopted ArgoCD at the massive scale.

Conclusion

In conclusion, ArgoCD and Flux both provide robust GitOps capabilities for managing infrastructure and applications on Kubernetes.

In selecting a GitOps operator, consider how factors like UI needs, multi-tenancy requirements, synchronization flexibility, and ecosystem integrations line up with your use case. Both ArgoCD and Flux are excellent choices - focus on which philosophy and feature set best matches your needs. We picked ArgoCD and in the next article would break down things like Git directory structure, dealing with scale, templating, etc - so stay tuned!