7. Connect to Oracle Health

Your service gets patient demographics from a REST service, the Mobile MVI Service. Many VA Mobile FHIR services also call other FHIR servers, most often the Oracle Health (Cerner) Millennium FHIR API, the electronic health record (EHR) that is replacing VistA at VA medical centers.

Over this page and the next, you add an Encounter search, which returns a patient’s encounters (visits) from Oracle Health. On this page, you prepare everything the search needs, without writing any Java code that calls Oracle Health yet:

You deploy cerner-mock, which stands in for Oracle Health, and load test data into it.
You register the FHIR client for Oracle Health, and add the settings and secrets it needs to get an access token.
You make sure the service refuses to start without those settings.

On the next page, you write the code that uses them.

What cerner-mock Is

You can’t reach Oracle Health from your workstation. In local deployments and integration tests, VA Mobile FHIR services such as cerner-fhir-adapter (cfa) and clinical-encounter-service (ces) use cerner-mock instead. cerner-mock is a lightweight FHIR server that behaves like the Oracle Health Millennium FHIR R4 API:

Unlike a WireMock stub, it holds real FHIR resources. You load Patients and Encounters into it, and it answers searches for them the way Oracle Health does.
Like Oracle Health, it requires an access token for every resource request, and serves its CapabilityStatement (/metadata) without one.
Oracle Health hosts many organizations, called tenants, and the tenant is part of every URL. cerner-mock’s test tenant is CM-VA-TID.

Add cerner-mock to the Local Deployment

In kubernetes/dev-with-dependencies/kustomization.yaml, add the cerner-mock component after WireMock, and the remove-resource-constraints component at the end:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
...
components:
...
- https://coderepo.mobilehealth.va.gov/scm/dhsss/wiremock.git//kubernetes?ref=v3.13.2&timeout=120s
- https://coderepo.mobilehealth.va.gov/scm/ces/cerner-mock.git//kubernetes/components/dev?ref=Release/1.16&timeout=120s
- ../components/dev
- ../components/remove-resource-constraints

cerner-mock asks Kubernetes to reserve a full CPU and 2 GiB of memory, which a local cluster may not have to spare. The generated remove-resource-constraints component removes the resource requests and limits from every deployment. Use it only for local deployments, as cfa and ces do. Your service, WireMock, and cerner-mock together use about 1.3 GB of memory.

Load Test Data into cerner-mock

When cerner-mock starts, it loads every FHIR resource and Bundle in the /opt/fhir/r4 directory and its subdirectories. You will put your test data in a ConfigMap and mount it there, as you did with the WireMock stubs.

Create the directory kubernetes/components/dev/cerner-mock-data and add three files. patient-10115V737384.json is PATIENT FIFTEEN’s record in Oracle Health. Its id, 90000015, is the Oracle Health id, and the ICN is one of its identifiers:

{
  "resourceType": "Patient",
  "id": "90000015",
  "identifier": [
    {
      "system": "urn:oid:2.16.840.1.113883.4.349",
      "value": "10115V737384"
    }
  ],
  "name": [
    {
      "family": "FIFTEEN",
      "given": [
        "PATIENT"
      ]
    }
  ],
  "gender": "male",
  "birthDate": "1953-04-01"
}

encounters-10115V737384.json is a Bundle with three encounters of PATIENT FIFTEEN, two finished and one planned. Their subject refers to the patient by its Oracle Health id:

{
  "resourceType": "Bundle",
  "type": "collection",
  "entry": [
    {
      "resource": {
        "resourceType": "Encounter",
        "id": "90001501",
        "status": "finished",
        "class": {
          "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
          "code": "AMB"
        },
        "subject": {
          "reference": "Patient/90000015"
        },
        "period": {
          "start": "2024-02-01T14:00:00Z",
          "end": "2024-02-01T14:30:00Z"
        }
      }
    },
    {
      "resource": {
        "resourceType": "Encounter",
        "id": "90001502",
        "status": "finished",
        "class": {
          "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
          "code": "AMB"
        },
        "subject": {
          "reference": "Patient/90000015"
        },
        "period": {
          "start": "2025-06-15T09:00:00Z",
          "end": "2025-06-15T09:45:00Z"
        }
      }
    },
    {
      "resource": {
        "resourceType": "Encounter",
        "id": "90001503",
        "status": "planned",
        "class": {
          "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
          "code": "AMB"
        },
        "subject": {
          "reference": "Patient/90000015"
        },
        "period": {
          "start": "2026-12-01T10:00:00Z"
        }
      }
    }
  ]
}

patients-10180V273016.json has two patients with the same ICN, 10180V273016. That shouldn’t happen, but it does happen in real data, and your service must not guess which patient is the right one:

{
  "resourceType": "Bundle",
  "type": "collection",
  "entry": [
    {
      "resource": {
        "resourceType": "Patient",
        "id": "90000080",
        "identifier": [
          {
            "system": "urn:oid:2.16.840.1.113883.4.349",
            "value": "10180V273016"
          }
        ],
        "name": [
          {
            "family": "EIGHTY",
            "given": [
              "PATIENT"
            ]
          }
        ]
      }
    },
    {
      "resource": {
        "resourceType": "Patient",
        "id": "90000081",
        "identifier": [
          {
            "system": "urn:oid:2.16.840.1.113883.4.349",
            "value": "10180V273016"
          }
        ],
        "name": [
          {
            "family": "EIGHTY",
            "given": [
              "PAT"
            ]
          }
        ]
      }
    }
  ]
}

PATIENT THIRTYTWO (10132V467385) has no record in Oracle Health on purpose, so you can see what a search for a patient without an Oracle Health record returns. cerner-mock keeps the ids in the files, so your tests can rely on them.

In kubernetes/components/dev/kustomization.yaml, generate a ConfigMap from the files and add a patch for the cerner-mock deployment:

...
configMapGenerator:
...
- name: mvi-wiremock-mappings
  ...
- name: cerner-mock-data
  behavior: create
  files:
  - cerner-mock-data/patient-10115V737384.json
  - cerner-mock-data/encounters-10115V737384.json
  - cerner-mock-data/patients-10180V273016.json
...
patches:
- path: deployment.yaml
- path: service.yaml
- path: wiremock-data/data-wiremock.yaml
- path: cerner-mock-data/data-cerner-mock.yaml
...

Then create the patch, kubernetes/components/dev/cerner-mock-data/data-cerner-mock.yaml. It mounts the ConfigMap into the cerner-mock container, in a subdirectory of /opt/fhir/r4:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: cerner-mock-v1
spec:
  template:
    spec:
      volumes:
      - name: cerner-mock-data
        configMap:
          name: cerner-mock-data
      containers:
      - name: cerner-mock
        volumeMounts:
        - name: cerner-mock-data
          mountPath: /opt/fhir/r4/fhirstarter

cerner-mock can also load resources while it runs, with its $load operation. The integration tests of fehr-fhir-service use it to create the data each test needs.

Register the FHIR Client

On page 3, you registered a REST client for the Mobile MVI Service. FHIR clients are registered the same way, with va.mobile.fhir.client.registration.<id> properties, and you get them from FhirClientManager with getClient("<id>"). Each registration also gets a health indicator named after it, which calls the server’s /metadata.

Update src/main/resources/application.properties to resemble the following. Besides the cerner FHIR client, it adds the Oracle Health token URL and credentials in place of the example setting, and adds cerner to the readiness group:

spring.application.name=fhir-starter-service-v1
server.servlet.context-path=/fhirstarter/v1

# FHIR Server path, relative to the context path. The FHIR base URL is the context path plus this path (without /*).
va.mobile.fhir.server.path=/fhir/*

management.endpoint.health.group.liveness.include=livenessState,diskSpace,ping
# The service is only ready when the Mobile MVI Service and Oracle Health are reachable (health indicators "mvi" and "cerner")
management.endpoint.health.group.readiness.include=readinessState,mvi,cerner

# Service Properties (see AppProperties)
# Oracle Health OAuth 2.0 token endpoint (Consul), and the client ID and secret of the service (Vault)
va.mobile.fhirstarter.cerner.token-url=${CERNER_TOKEN_URL:}
va.mobile.fhirstarter.cerner.client-id=${CERNER_CLIENT_ID:}
va.mobile.fhirstarter.cerner.client-secret=${CERNER_CLIENT_SECRET:}

# Mobile MVI Service: patient demographics and identifiers, used through RestClientManager.getClient("mvi")
va.mobile.restclient.registration.mvi.url=http://mobile-mvi-service-v1:8080/mvi/v1

# Oracle Health (Cerner) Millennium FHIR R4 API, used through FhirClientManager.getClient("cerner")
va.mobile.fhir.client.registration.cerner.url=${CERNER_FHIR_URL:}
va.mobile.fhir.client.registration.cerner.factory.connect-timeout=2s
va.mobile.fhir.client.registration.cerner.factory.socket-timeout=30s

# Disable Spring Cloud Discovery
spring.cloud.discovery.enabled=false

# Vault Properties
spring.cloud.vault.uri=${VAULT_ADDR:}
spring.cloud.vault.authentication=${VAULT_AUTH:APPROLE}
spring.cloud.vault.app-role.role-id=${NAMESPACE:}-read
spring.config.import=optional:vault://${VAULT_MOUNT_PATH:}/${NAMESPACE:}/${APP_NAME:fhir-starter-service}/${APP_VERSION:v1}

${CERNER_FHIR_URL:} reads the environment variable CERNER_FHIR_URL, and is empty when the variable isn’t set. In SQA and production, CERNER_FHIR_URL and CERNER_TOKEN_URL come from Consul, and CERNER_CLIENT_ID and CERNER_CLIENT_SECRET come from Vault.
The timeouts keep a slow Oracle Health from tying up your request threads: two seconds to connect, and 30 seconds for a response.
With cerner in the readiness group, Kubernetes only sends requests to your service while Oracle Health answers the health check. The health check calls /metadata, which needs no token.

Point the dev profile at cerner-mock. Add the following to src/main/resources/application-dev.properties:

# Local deployments use cerner-mock instead of Oracle Health
va.mobile.fhir.client.registration.cerner.url=http://cerner-mock-v1:8080/cerner-mock/v1/r4/CM-VA-TID
va.mobile.fhirstarter.cerner.token-url=http://cerner-mock-v1:8080/cerner-mock/v1/tenants/CM-VA-TID/protocols/oauth2/profiles/smart-v1/token

The client ID and secret are secrets, so they don’t go in a properties file. In local deployments, they come from kubernetes/components/secrets/application-secrets.env, which the dev component passes to your service as environment variables. Replace the contents of the file with the following. (The generated file doesn’t end with a line break, so text appended with echo >> would end up on the commented API_CLIENTS_V1 line.)

#################################################
# Specify test secrets (DO NOT USE REAL VALUES) #
#################################################
# API_CLIENTS_V1=[{"name": "fhir-starter-service-v1","id": "gov.va.mobile.fhir-starter-service.v1","secret": "1234567"}]

# Built-in fake client of cerner-mock
CERNER_CLIENT_ID=deadbeef-3cd4-4c63-a6c4-903d9bac35a5
CERNER_CLIENT_SECRET=ces_sekret

These are cerner-mock’s built-in client ID and secret. They are fake values that only work with cerner-mock, which doesn’t accept other clients today. application-secrets.env is only used by local deployments. Real secrets never go in the repository; they are stored in Vault.

Replace the Example Setting with the Cerner Settings

The generated AppProperties has an example setting. Replace it with a nested class for the Oracle Health settings. Update AppProperties.java to the following:

package gov.va.mobile.fhirstarter.v1;

import jakarta.validation.Valid;
import jakarta.validation.constraints.NotEmpty;
import jakarta.validation.constraints.NotNull;
import lombok.Getter;
import lombok.Setter;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.context.properties.NestedConfigurationProperty;
import org.springframework.validation.annotation.Validated;

import java.time.Duration;
import java.util.List;

/**
 * Configuration properties for fhir-starter-service (FHIR Service), bound from {@code va.mobile.fhirstarter.*}.
 *
 * <p>Properties are validated at startup ({@link Validated}), so a missing or invalid value stops the service from
 * starting instead of failing on the first request.
 *
 * @since 1.0
 */
@Getter
@Setter
@Validated
@ConfigurationProperties("va.mobile.fhirstarter")
public class AppProperties {

    /**
     * Settings of the Oracle Health (Cerner) Millennium FHIR API, bound from {@code va.mobile.fhirstarter.cerner.*}.
     */
    @Valid
    @NestedConfigurationProperty
    private final Cerner cerner = new Cerner();

    /**
     * Settings of the Oracle Health (Cerner) Millennium FHIR API.
     */
    @Getter
    @Setter
    public static class Cerner {

        /**
         * URL of the OAuth 2.0 token endpoint ({@code CERNER_TOKEN_URL}).
         */
        @NotEmpty
        private String tokenUrl;

        /**
         * Client ID of the service, read from Vault ({@code CERNER_CLIENT_ID}).
         */
        @NotEmpty
        private String clientId;

        /**
         * Client secret of the service, read from Vault ({@code CERNER_CLIENT_SECRET}).
         */
        @NotEmpty
        private String clientSecret;

        /**
         * Scopes requested with each token.
         */
        @NotEmpty
        private List<String> scopes = List.of("system/Patient.read", "system/Encounter.read");

        /**
         * How long before the current token expires a new token is requested.
         */
        @NotNull
        private Duration tokenRefreshMargin = Duration.ofSeconds(30);
    }
}

Spring binds va.mobile.fhirstarter.cerner.token-url to cerner.tokenUrl, and so on. @Valid makes Spring validate the nested class too.
scopes and tokenRefreshMargin have defaults, so they need no properties. You can still override them, for example with va.mobile.fhirstarter.cerner.token-refresh-margin=1m.

Document the new environment variables in metadata.yaml. Replace the contents of the file with the following, which lists the two URLs under consul and the two secrets under vault, in place of the EXAMPLE_SETTING entry and the commented vault example:

documentation:
  jiraProjectKey: CKM
  # API Gateway registration of the service: staff, veteran, both (the filegen default) or none
  # apigw:
  #   registration: staff
  consul:
    required:
    - name: JWT_PUBLIC_KEY
      comment: Public key to verify JWTs
    - name: CERNER_FHIR_URL
      comment: Base URL of the Oracle Health (Cerner) Millennium FHIR R4 API (va.mobile.fhir.client.registration.cerner.url)
    - name: CERNER_TOKEN_URL
      comment: URL of the Oracle Health OAuth 2.0 token endpoint (va.mobile.fhirstarter.cerner.token-url)
  vault:
  - name: CERNER_CLIENT_ID
    comment: Client ID of the service, registered with Oracle Health (va.mobile.fhirstarter.cerner.client-id)
  - name: CERNER_CLIENT_SECRET
    comment: Client secret of the service, registered with Oracle Health (va.mobile.fhirstarter.cerner.client-secret)
  po: 'TODO: Product Owner Name'

The build generates the deployment instructions, docs/dibr.md, from this file. After your next build, its Vault Configuration section lists the two secrets.

Make Every Profile Start

Run the unit tests:

./mvnw test

SpringSmokeTest now fails for every profile. Each failure explains why the service didn’t start, for example:

APPLICATION FAILED TO START
...
Binding to target gov.va.mobile.fhirstarter.v1.AppProperties failed:

    Property: va.mobile.fhirstarter.cerner.clientId
    Value: ""
    Reason: must not be empty

    Property: va.mobile.fhirstarter.cerner.clientSecret
    Value: ""
    Reason: must not be empty

    Property: va.mobile.fhirstarter.cerner.tokenUrl
    Value: ""
    Reason: must not be empty

That’s what you want: a service with a missing setting stops at startup, where the problem is obvious, instead of failing on its first request. The unit tests run without the environment variables of a deployment, so even the dev profile is missing the client ID and secret. The framework also refuses to start when the URL of a FHIR client registration is empty.

SpringSmokeTest only checks that the application starts, so give it values for the four settings. Update its @SpringBootTest annotation:

@SpringBootTest(properties = {
        "spring.cloud.vault.enabled=false",
        "va.mobile.fhir.client.registration.cerner.url=http://localhost/r4",
        "va.mobile.fhirstarter.cerner.token-url=http://localhost/token",
        "va.mobile.fhirstarter.cerner.client-id=smoke-test",
        "va.mobile.fhirstarter.cerner.client-secret=smoke-test"})

Run ./mvnw test again; the tests pass.

Try It

You changed files under kubernetes, so stop skaffold dev, run ./scripts/localize.sh force, and start skaffold dev --rpc-port 50051 --port-forward=true again. Wait until both your service and cerner-mock are running.

cerner-mock takes longer to start than your service. Until it’s ready, the service log may show long Health check failed warnings with stack traces, such as Connect to http://cerner-mock-v1:8080 failed: Connection refused. They are expected, and stop when cerner-mock is up and Skaffold prints Deployments stabilized.

When the pods are ready, check the health of your service with curl -s http://localhost:8081/actuator/health. The cerner component is UP, with the URL of its health check, http://cerner-mock-v1:8080/cerner-mock/v1/r4/CM-VA-TID/metadata.

Before you write any code, look at cerner-mock directly, to see the requests your service will send. This takes three terminals: the first keeps running skaffold dev. In a second terminal, forward a local port to cerner-mock. Like skaffold dev, this command keeps running until you press Ctrl-C:

kubectl port-forward -n fhir-starter-service-test svc/cerner-mock-v1 8090:8080

Then, in a third terminal, request an access token with the client ID and secret from application-secrets.env, and save it in CERNER_TOKEN:

export CERNER_TOKEN=$(curl -s -u 'deadbeef-3cd4-4c63-a6c4-903d9bac35a5:ces_sekret' -d 'grant_type=client_credentials&scope=system/Patient.read,system/Encounter.read' 'http://localhost:8090/cerner-mock/v1/tenants/CM-VA-TID/protocols/oauth2/profiles/smart-v1/token' | jq -r .access_token)

jq -r .access_token picks the token out of the JSON response. Now search for the encounters of the Oracle Health patient 90000015:

curl -s -H "Authorization: Bearer $CERNER_TOKEN" 'http://localhost:8090/cerner-mock/v1/r4/CM-VA-TID/Encounter?subject=Patient/90000015' | jq

The response is a Bundle with PATIENT FIFTEEN’s three encounters, as you loaded them. On the next page, your service sends the same requests. When you’re done, press Ctrl-C in the terminal that runs kubectl port-forward.

Update the Tests

To check your tests, compare them with the test classes in the "FHIR starter page 7" commit of the reference branch; see When You Get Stuck.

In HealthCheckITCase, check that the cerner health indicator is up too:

                .hasComponentsUp("mvi", "cerner")

Review Your Configuration

You have now added settings in several places. Each place has a purpose:

File What belongs there

File	What belongs there
`src/main/resources/application.properties`	Settings that are the same in every environment, and placeholders for the ones that aren’t, such as `${CERNER_FHIR_URL:}`
`src/main/resources/application-dev.properties`	Settings for local deployments and integration tests: the WireMock and cerner-mock URLs, debug logging, and turning off Vault and the audit client
`kubernetes/components/dev/overrides.env`	Environment variables for local deployments, such as `SPRING_PROFILES_ACTIVE=dev`. Each environment has its own `overrides.env`, for example `kubernetes/sqa/overrides.env`.
`kubernetes/components/secrets/application-secrets.env`	Fake secrets for local deployments, passed to your service as environment variables
`metadata.yaml`	Documentation of the environment variables that SQA and production must provide: Consul settings and Vault secrets

src/main/resources/application.properties

Settings that are the same in every environment, and placeholders for the ones that aren’t, such as ${CERNER_FHIR_URL:}

src/main/resources/application-dev.properties

Settings for local deployments and integration tests: the WireMock and cerner-mock URLs, debug logging, and turning off Vault and the audit client

kubernetes/components/dev/overrides.env

Environment variables for local deployments, such as SPRING_PROFILES_ACTIVE=dev. Each environment has its own overrides.env, for example kubernetes/sqa/overrides.env.

kubernetes/components/secrets/application-secrets.env

Fake secrets for local deployments, passed to your service as environment variables

metadata.yaml

Documentation of the environment variables that SQA and production must provide: Consul settings and Vault secrets

A placeholder such as ${CERNER_TOKEN_URL:} reads an environment variable, with the default value after the colon. Here the default is empty, which @NotEmpty rejects at startup.
Spring also maps environment variables to properties directly, by a convention called relaxed binding. For example, the commented-out LOGGING_LEVEL_GOV_VA_MOBILE=DEBUG in overrides.env would set logging.level.gov.va.mobile=DEBUG.
Everything under kubernetes/base applies to every environment. Settings and data for local deployments only belong in the dev component, kubernetes/components/dev, like your WireMock stubs and cerner-mock data.

Run the Build

./mvnw clean install -Pwith-skaffold

Commit Your Work

git add .
git commit -m "Deploy cerner-mock and configure the Oracle Health FHIR client"
git push

See It in a Real Service

clinical-encounter-service: src/main/resources/application.properties, which registers FHIR clients for cerner-fhir-adapter and vista-fhir-adapter
fehr-fhir-service: src/test/java/gov/va/mobile/fehr/v1/utils/CernerMockUtils.java, which loads test data into cerner-mock at runtime, including two patients with the same identifier

You’re Done When

cerner-mock runs in your local cluster with your test data, and you have searched it directly with an access token
The cerner health indicator is UP and part of the readiness group
SpringSmokeTest passes for every profile
You can explain where each setting and secret of the service is defined
All tests pass, and your work is committed and pushed

Continue to 8. Search Encounters in Oracle Health to search encounters in Oracle Health.