Importing untrusted TLS certificates to Che

Che components communications with external services are encrypted with TLS. They require TLS certificates signed by trusted Certificate Authorities (CA). Therefore, you must import into Che all untrusted CA chains in use by an external service such as:

  • A proxy

  • An identity provider (OIDC)

  • A source code repositories provider (Git)

  • A parent devfile referenced via URI with untrusted TLS

Che uses labeled ConfigMaps in Che namespace as sources for TLS certificates. The ConfigMaps can have an arbitrary amount of keys with an arbitrary amount of certificates each. All certificates are mounted into:

  • /public-certs location of Che server and dashboard pods

  • /etc/pki/ca-trust/extracted/pem locations of workspaces pods

Configure the CheCluster custom resource to disable CA bundle mounting at /etc/pki/ca-trust/extracted/pem. The certificates will instead be mounted at /public-certs to keep the behaviour from the previous version.

Configure the CheCluster Custom Resource in order to disable the mounting of the CA bundle under the path /etc/pki/ca-trust/extracted/pem. Certificates will be mounted under the path /public-certs in this case.

spec:
  devEnvironments:
    trustedCerts:
      disableWorkspaceCaBundleMount: true

On OpenShift cluster, Che operator automatically adds Red Hat Enterprise Linux CoreOS (RHCOS) trust bundle into mounted certificates.

When certificates are imported, the Che operator automatically configures the DevWorkspace Operator to trust these certificates by setting the tlsCertificateConfigmapRef field in the DevWorkspace Operator configuration to reference the ca-certs-merged ConfigMap. This enables the DevWorkspace Operator to trust the certificates when resolving parent devfile references via URI that use untrusted TLS certificates.

Prerequisites

  • An active kubectl session with administrative permissions to the destination Kubernetes cluster. See Overview of kubectl.

  • The eclipse-che namespace exists.

  • For each CA chain to import: the root CA and intermediate certificates, in PEM format, in a ca-cert-for-che-<count>.pem file.

Procedure

  1. Concatenate all CA chains PEM files to import, into the custom-ca-certificates.pem file, and remove the return character that is incompatible with the Java truststore.

    $ cat ca-cert-for-che-*.pem | tr -d '\r' > custom-ca-certificates.pem

  2. Create the custom-ca-certificates ConfigMap with the required TLS certificates:

    $ kubectl create configmap custom-ca-certificates \
    --from-file=custom-ca-certificates.pem \
    --namespace=eclipse-che

  3. Label the custom-ca-certificates ConfigMap:

    $ kubectl label configmap custom-ca-certificates \
    app.kubernetes.io/component=ca-bundle \
    app.kubernetes.io/part-of=che.eclipse.org \
    --namespace=eclipse-che

  4. Deploy Che if it hasn’t been deployed before. Otherwise, wait until the rollout of Che components finishes.

  5. Restart running workspaces for the changes to take effect.

Verification steps

  1. Verify that the ConfigMap contains your custom CA certificates. This command returns CA bundle certificates in PEM format:

    kubectl get configmap \
    --namespace=eclipse-che \
    --output='jsonpath={.items[0:].data.custom-ca-certificates\.pem}' \
    --selector=app.kubernetes.io/component=ca-bundle,app.kubernetes.io/part-of=che.eclipse.org

  2. Verify in the Che server logs that the imported certificates count is not null:

    kubectl logs deploy/che --namespace=eclipse-che \
    | grep tls-ca-bundle.pem

  3. Start a workspace, get the namespace name in which it has been created: <workspace_namespace>, and wait for the workspace to be started.

  4. Verify that the ca-certs-merged ConfigMap contains your custom CA certificates. This command returns Che CA bundle certificates in PEM format:

    kubectl get configmap ca-certs-merged \
    --namespace=<workspace_namespace> \
    --output='jsonpath={.data.tls-ca-bundle\.pem}'

  5. Verify that the DevWorkspace Operator is configured to trust the imported certificates. This command returns the TLS certificate ConfigMap reference in the DevWorkspace Operator configuration:

    kubectl get devworkspaceoperatorconfig \
    --namespace=eclipse-che \
    devworkspace-config \
    --output='jsonpath={.config.routing.tlsCertificateConfigmapRef}'

    The output should show the reference to the ca-certs-merged ConfigMap:

    {"name":"ca-certs-merged","namespace":"<prod-namespace>"}

  6. Verify that the workspace pod mounts the ca-certs-merged ConfigMap:

    kubectl get pod \
    --namespace=<workspace_namespace> \
    --selector='controller.devfile.io/devworkspace_name=<workspace_name>' \
    --output='jsonpath={.items[0:].spec.volumes[0:].configMap.name}' \
    | grep ca-certs-merged

  7. Get the workspace pod name <workspace_pod_name>:

    kubectl get pod \
    --namespace=<workspace_namespace> \
    --selector='controller.devfile.io/devworkspace_name=<workspace_name>' \
    --output='jsonpath={.items[0:].metadata.name}' \

  8. Verify that the workspace container has your custom CA certificates. This command returns Che CA bundle certificates in PEM format:

    kubectl exec <workspace_pod_name> \
    --namespace=<workspace_namespace> \
    -- cat /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem

    Or if disableWorkspaceCaBundleMount set to true:

    kubectl exec <workspace_pod_name> \
    --namespace=<workspace_namespace> \
    -- cat /public-certs/tls-ca-bundle.pem
Additional resources