epilot API: providing data from other systems via REST

The epilot API is how information from your surrounding systems ends up in epilot. This article explains how data from other CRMs, databases and legacy systems is provided through a REST API, how it lands in the right place inside epilot, and which rules keep the integration dependable once it is live.

I am Marcel R. G. Berger, a software developer and architect with more than 20 years of experience. I work with epilot as an external consultant and developer for apps and customisations. This piece is for IT leads at grid operators, municipal utilities and energy suppliers whose data does not live in epilot alone.

How does data from surrounding systems reach epilot through the epilot API?

Data from surrounding systems reaches epilot through a small dedicated service that sits between the two. It reads the source, translates the data into epilot’s data model and writes it through the platform’s public APIs.

epilot offers several routes for this. The integrations overview in the epilot documentation lists the Integration Toolkit for ERP systems, the Batch API for many calls in one request, 3rd Party Journeys for external forms, webhooks for real-time events, and the public APIs for custom integrations with full control.

For data from another CRM or your own database, the epilot Entity API is usually at the centre. The documentation describes it as the primary data layer for business objects in epilot, covering schemas, CRUD, search, relations and an activity log.

A typical service therefore takes care of four things:

• reading data from the source, for example through its REST API or through a dedicated interface in front of the database
• mapping fields, converting formats and checking required values
• finding, creating or updating the matching entity in epilot
• detecting failures, retrying and logging in a traceable way

How such a service is built as a mapper with Quarkus is covered in detail in mappers between epilot and surrounding systems with Quarkus.

Pull or push: which pattern fits your data?

The right pattern depends on where a change becomes known first and how current the information in epilot needs to be. Most projects end up using both, depending on the kind of data.

Push means the surrounding system reports a change and the service writes it to epilot straight away. Pull means the service fetches data, either on a schedule or exactly when epilot needs it.

Pattern	Trigger	Good fit for	Watch out for
Push from the source system	a change in the CRM or line-of-business system	data that has to be current in epilot quickly	the source has to be able to report changes
Scheduled pull	a fixed interval in the service	master data that rarely changes	data is slightly stale between two runs
Pull on demand	an event in epilot sent as a webhook	information needed for a single case only	the source has to be reachable at that moment
Polling from epilot	the external system polls a queue	systems that cannot accept inbound calls	polling interval and processing on the external side

Pull on demand usually follows the same flow. Something happens in epilot, epilot sends a webhook to the service, the service fetches the data from the surrounding system and writes the result back to epilot.

For the opposite direction, the epilot Integration Toolkit describes Pollable Outbound, where an ERP polls a queue instead of receiving webhooks. That helps with systems behind a firewall that do not accept calls from outside.

The rate limits page recommends subscribing to events through webhooks rather than polling endpoints. For the direction from epilot to your service, push via webhooks is the norm.

Why put a dedicated REST API in front of a database or legacy system?

A dedicated REST API is worth it because it decouples epilot, and every other consumer, from the internal structure of the database. It exposes only the fields that are needed and stays stable even when tables change.

Many legacy systems offer only direct database access or responses far too technical for epilot. A small service in front of them solves that.

Such a service layer does several jobs:

• It returns business objects rather than table rows, such as one customer summary instead of five joined tables.
• It exposes approved fields only and holds everything else back.
• It secures access, regardless of how the database itself is protected.
• It can be versioned, tested and documented with OpenAPI.

In Quarkus such an endpoint is a plain class. The Quarkus guide to writing REST services covers the basics, and the documentation annotations come from MicroProfile OpenAPI.

@Path("/v1/customers")
@Tag(name = "Customers")
@Produces(MediaType.APPLICATION_JSON)
public class CustomerResource {

    @Inject
    CustomerRepository repository;

    @GET
    @Path("/{customerNumber}")
    @Operation(summary = "Returns the fields required for one customer")
    public Response byNumber(@PathParam("customerNumber") String customerNumber) {
        return repository.findSummary(customerNumber)
                .map(summary -> Response.ok(summary).build())
                .orElse(Response.status(Response.Status.NOT_FOUND).build());
    }
}

CustomerRepository and the returned object are classes of the individual project. The principle is what matters: the response contains only what the other side needs, and the version sits in the path.

Where the line between configuration in epilot and custom development usually falls is explained in when low-code in epilot is no longer enough.

How does a service write information back to epilot?

A service writes information back through epilot’s public APIs, in most cases the Entity API. The data from the surrounding system then shows up directly on the case, the contact or the contract in epilot.

The Entity API documentation lists these operations, among others:

Operation	Method and path as documented	Used for
createEntity	POST /v1/entity/{schema}	creating a new entity
getEntity	GET /v1/entity/{schema}/{id}	reading one entity
updateEntity	PUT /v1/entity/{schema}/{id}	updating an existing entity
listEntities	POST /v1/entity:list	filtered lookup, for example by customer number
addRelations	POST /v1/entity/{schema}/{id}/relations	linking entities to each other

According to the documentation, every change made through the Entity API automatically creates an entry in the activity log.

For many records at once there is the Batch API. It accepts multiple calls in one request, processes them asynchronously and returns the results via a job ID. The example in the documentation shows contact upserts based on a unique key.

With the Quarkus REST Client the other side is described as a Java interface. Address and token live in configuration, not in the code.

@RegisterRestClient(configKey = "epilot-entity")
@ClientHeaderParam(name = "Authorization", value = "Bearer ${epilot.access-token}")
@Path("/v1/entity")
public interface EpilotEntityClient {

    @PUT
    @Path("/{schema}/{id}")
    @Consumes(MediaType.APPLICATION_JSON)
    Response updateEntity(@PathParam("schema") String schema,
                          @PathParam("id") String id,
                          Map<String, Object> attributes);
}

In the configuration, quarkus.rest-client.epilot-entity.url and epilot.access-token point to environment variables. How an operation treats fields that are not sent is something to check in the Entity API reference before going live.

How does a service trigger events in other systems?

A service triggers events in other systems by reacting to an event from epilot and calling the target system’s interface. epilot supplies the event, and the service turns it into the call the other system expects.

The integrations overview explains that epilot emits business events for what happens in your organisation. Core events ship with the platform, while custom events are defined by your organisation from its own attributes. Both are consumed through webhooks, automations or the Event Catalog API.

The core events page lists fields every event carries, including a unique event ID, the time, the event name and its version. The ID lets a service recognise an event delivered twice and process it only once.

Typical targets for such events are:

• another CRM in which a contact or case is created
• a line-of-business system that should start an order or a check
• a database that records a status for reporting
• a messaging interface that notifies further services

This is also how I work with a grid operator in Hesse. Many processes there run outside epilot, and several third-party systems are connected through individually developed interfaces. More on that in the in-practice section on the home page.

How do OpenAPI and versioning protect the API contract?

OpenAPI and versioning protect the contract between systems because both sides know, in writing and in a machine-readable form, which fields, types and error codes apply. Changes become visible before they surface in production.

Quarkus generates the OpenAPI description straight from the code. According to the Quarkus guide to OpenAPI and Swagger UI, the quarkus-smallrye-openapi extension serves the document at /q/openapi, with Swagger UI at /q/swagger-ui.

A few simple rules have proven themselves for versioning:

1. The major version goes in the path, for example /v1/customers.
2. New, optional fields are allowed within a version.
3. Renaming, removing or changing the type of a field means a new version.
4. Old and new versions run side by side for an agreed transition period.

The same rules apply in the other direction. epilot keeps its own API changelog, which is worth a look whenever the integration evolves.

How is a REST API interface tested?

A REST API interface is tested at several levels: the mapping with unit tests, the service with integration tests and the contract against the OpenAPI description. The Quarkus guide to testing your application shows how @QuarkusTest boots the application and calls endpoints with REST Assured.

Each mapping comes with sample data from the source and the expected result for epilot. Responses from the surrounding systems and from the epilot API are simulated in tests, so no test ever runs against real customer data. Failure cases such as 404, 429 or timeouts are explicitly part of the suite.

How do authentication, secrets and rate limits fit together?

Authentication, secrets and rate limits belong together because all three decide whether a service can talk to epilot securely and without interruption. None of them belongs in the source code.

According to the access tokens documentation, calls to the epilot APIs are authorised with an access token in the Authorization: Bearer header. Tokens are managed in epilot under Settings, Access Tokens, are shown only once, and can be given an expiry, which the documentation recommends. Tokens can also be restricted to read-only access.

That leads to a few clear rules:

• Every integration gets its own token with exactly the permissions it needs.
• A service that only reads gets a read-only token.
• Tokens, addresses and passwords for surrounding systems come from environment variables or a secret store, never from the repository. The Quarkus configuration reference shows how values are read from different sources.
• Tokens are rotated regularly and revoked immediately if anything looks wrong.

On rate limits the documentation is unambiguous. The rate limits page states that the API responds with HTTP 429 when a limit is exceeded, and the Retry-After header gives the seconds until the quota resets. Limits apply per organisation and minute. It lists 500 requests per minute for Entity API mutations and 5,000 for entity searches, with GET requests exempt. Figures like these can change, so check the current page first.

A service should therefore wait for the stated time on a 429 before retrying. For larger volumes it bundles calls through the Batch API instead of sending each change on its own.

How fresh does the data need to be, and when does caching help?

How fresh the data needs to be is decided by the process, not by the technology. For each kind of data you agree how old a piece of information in epilot may be at most, and that tells you whether a cache is acceptable.

Kind of data	Typical requirement	Approach
Status of an ongoing case	current at the moment it is displayed	no cache, push or pull on demand
Customer master data	current after every change	push on change
Reference data and catalogues	rarely changes	cache with a fixed expiry
Reports	a snapshot as of a given date is enough	scheduled pull

The rate limits page itself recommends caching GET responses to avoid unnecessary calls. The same applies to your surrounding systems.

In Quarkus the cache extension is all you need. The Quarkus guide to application data caching describes @CacheResult and per-cache expiry.

@ApplicationScoped
public class ReferenceDataLookup {

    @Inject
    @RestClient
    SourceSystemClient client;

    @CacheResult(cacheName = "reference-data")
    public ReferenceData byCode(String code) {
        return client.referenceData(code);
    }
}

With quarkus.cache.caffeine."reference-data".expire-after-write=10M an entry stays valid for ten minutes. The value is a placeholder and is agreed with the business side for each kind of data.

Which data should flow from surrounding systems into epilot?

Only the data the process in epilot actually needs should flow into it. That is the GDPR principle of data minimisation, and it also makes interfaces smaller and easier to review.

In practice this means:

• Every field has a documented reason for being transferred.
• The service layer in front of the database exposes approved fields only.
• Logs contain identifiers, not complete personal records.
• Test data is synthetic or anonymised, never a copy of real customer data.

According to the documentation, epilot also offers anonymised access tokens, where personal data in Entity API responses is replaced with pseudonyms. The documentation itself calls this best effort, and custom attributes need to be classified accordingly.

The data stays in Europe. Where I process personal data on your behalf we sign a data processing agreement, and professional liability insurance is in place. The contracting party is DigitalFreedom Global LLC. Grid operators can opt for German law on request, otherwise Wyoming law applies. Invoices are issued in EUR, with reverse charge for EU business clients after a VIES check. The details are in the contracting party section on the home page.

How do you get started with a REST API interface for epilot?

You start with a free intro call in which we go through your surrounding systems, kinds of data and workflows. Afterwards it is clear which data epilot needs and which pattern fits.

1. Intro call: free and without obligation, 30 minutes on Microsoft Teams.
2. Individual quote: tailored to your systems, data volumes and requirements for freshness and operation.
3. Delivery: service layer, connection to the epilot API, tests and go-live with your systems.

The code belongs to you. On request I hand it over to your own repository, or I look after operation and further development under a maintenance contract. Why that matters is explained in who owns the integration code.

I work alongside epilot’s consulting team, not instead of it. epilot remains your point of contact for the platform, and I pick up where classic development is needed. The services section on the home page lists what that covers.

Everything I offer is based on an individual quote. More answers are in the frequently asked questions on the home page, and you can pick a slot directly in the booking calendar.