Gitops

how to work with gitops and argocd

Argocd password

You can obtain argocd admin password from stacks/eks outputs:

aws-vault exec igaawi/sso/administrators
cd stacks/eks
export ENV=prod
tg output --json | jq -r .argocd.value

{
  "password": "HIDDEN",
  "username": "admin"
}

then proceed to argocd

You can use taskfile argocd targets:

Command	Description
`argocd:aas`	turn on sync for the argocd app, task `argocd:aas -- apps backend`
`argocd:aau`	turn off sync for the argocd app, task `argocd:aau -- apps backend`

or you can use any argocd cli commands:

argocd app list -l argocd.argoproj.io/instance=apps -o name
argo-cd/backend
argo-cd/frontend
argo-cd/docs

.skipargocd

If you don’t want automatic shell provision for argocd via direnv you may want to put empty .skipargocd in the root of the kubernetes-aws repo.

Taskfile

We’re using Taskfile to simplify some heavy-lifting. Install it so you can run commands below.

Encrypt HELM secrets

In order to encrypt any secret using aws kms, perform the following steps

Go to your kubernetes-aws repository, and run the following taskfile command:

1
2
3
4


cd igaawi-kubernetes-aws
export ENV=prod
task helm:encrypt -- YOURSECRET
ref+awskms://AQICAHgAHnIOPWdyAGEfn3I...2DAPUFSlAY63pjsm34DNyc_vza?key=alias%2Figaawi-prod-eks-helm-secrets-kms-key

Where YOURSECRET is the actual value you want to encrypt
Then just insert entire line from stdout into desired secret in your values.yaml or values.ci.yaml files as needed

here is an example of that secret in the values.ci.yaml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


```yaml
secrets:
  githubAuthToken:
    annotations:
      devops/expiration: 12/30/2025 # mm/dd/yyyy
      devops/description: |
        used in git-clone tekton task
        PAT name: igaawi-tekton-token
        info: https://github.com/organizations/saritasa-nest/settings/personal-access-tokens/<TOKEN_ID>
          - created by udaltsovra
          - created on <DATE>
          - expires on <DATE>
        regenerate: https://github.com/settings/personal-access-tokens/<TOKEN_ID> (by udaltsovra)
stringData:
  password: ref+awskms://AQICAHipefxvxPDbk8rz34wqZfRH...ATe-Dzo?key=alias%2Figaawi-prod-eks-helm-secrets-kms-key
```

Shell History

You don’t want your unencrypted secrets to be inserted in your history, therefore adjust your ZSH by adding HISTORY_IGNORE to be HISTORY_IGNORE="(ls|cd|pwd|exit|cd|vi|task helm:encrypt)*"

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


HISTFILE=~/.histfile
HISTSIZE=1000000
SAVEHIST=1000000
HISTCONTROL=ignoreboth:erasedups

<!-- prettier-ignore-start -->
HISTORY_IGNORE="(ls|cd|pwd|exit|cd|vi|task helm:encrypt)*"

setopt EXTENDED_HISTORY      # Write the history file in the ':start:elapsed;command' format.
setopt INC_APPEND_HISTORY    # Write to the history file immediately, not when the shell exits.
setopt SHARE_HISTORY         # Share history between all sessions.
setopt HIST_IGNORE_DUPS      # Do not record an event that was just recorded again.
setopt HIST_IGNORE_ALL_DUPS  # Delete an old recorded event if a new event is a duplicate.
setopt HIST_IGNORE_SPACE     # Do not record an event starting with a space.
setopt HIST_SAVE_NO_DUPS     # Do not write a duplicate event to the history file.
setopt HIST_VERIFY           # Do not execute immediately upon history expansion.
setopt APPEND_HISTORY        # append to history file (Default)
setopt HIST_NO_STORE         # Don't store history commands
setopt HIST_REDUCE_BLANKS    # Remove superfluous blanks from each command line being added to the history.

Annotations

If your secret was generated somewhere, add that information into annotations.devops/description
If your secret expires, add that info in annotations.devops/expiration
Include link to regenerate the secret in the dedicated annotation or description

Here is an example. Include as much info as possible for other engineers to have a breeze maintenance.

annotations:
  devops/expiration: 12/30/2025 # mm/dd/yyyy
  devops/description: |
    used in git-clone tekton task
    PAT name: igaawi-tekton-token
    info: https://github.com/organizations/saritasa-nest/settings/personal-access-tokens/<TOKEN_ID>
      - created by udaltsovra
      - created on <DATE>
      - expires on <DATE>
    regenerate: https://github.com/settings/personal-access-tokens/<TOKEN_ID> (by udaltsovra)

Render argocd manifest

cd igaawi-kubernetes-aws
export ENV=prod
task helm:app -- apps docs
task helm:app -- config promtail

If you want to apply them, then, turn off root sync (i.e. apps or addons) and:

task helm:app -- apps docs | k apply -f-
task helm:app -- config promtail | k apply -f-

Decrypt HELM encrypted values

Commands above will render values without the secrets, if you want to see the secret in open text you can do:

task helm:app reveal=true -- apps backend

You can also decrypt the encrypted values using vals cli as shown below:

AWS_PROFILE=igaawi/sso/prod/administrators \
  vals eval -f apps/prod/values.ci.yaml \
  | yq

If you get the following error:

AWS_PROFILE=igaawi/sso/prod/administrators \
  vals eval -f apps/prod/values.ci.yaml
expand awskms://AQICAHipefxvxPDbk8rz34wqZfRH...ATe-Dzo?key=alias%2Figaawi-prod-eks-helm-secrets-kms-key: refresh cached SSO token failed, unable to refresh SSO token, InvalidGrantException:
{
  RespMetadata: {
    StatusCode: 400,
    RequestID: "d3e2ef31-ce1e-46c7-8a53-905ab62aeb26"
  },
  Error_: "invalid_grant",
  Error_description: "Invalid refresh token provided"
}

That means you need to login into your SSO V2 account, which you can do by:

aws sso login --profile saritasa/v2/administrators

Attempting to automatically open the SSO authorization page in your default browser.
If the browser does not open or you wish to use a different device to authorize this request, open the following URL:

https://device.sso.us-west-2.amazonaws.com/

Then enter the code:

WKNN-QJQQ
Successfully logged into Start URL: https://saritasa.awsapps.com/start

Disable argocd app sync

You may want to disable sync of any apps within argocd, you can use the following taskfile target:

task argocd:aau -- apps backend
application [apps] sync is turned off
application [backend] sync is turned off

task argocd:aas -- addons
application [addons] sync is turned off

You can pass as many arguments as you want. Keep in mind that there are 2 root apps: [addons] & [apps]

You want to disable their sync if you want to disable sync of the child app, otherwise it will be rolled back.

Enable argocd app sync

You may want to enable sync of any apps within argocd, you can use the following taskfile target:

task argocd:aas -- apps backend
application [apps] sync is turned on
application [backend] sync is turned on

task argocd:aas -- addons
application [addons] sync is turned on

You can pass as many arguments as you want.