Install EPM-UL using your certificates

This topic describes how to install and configure Endpoint Privilege Management for Unix and Linux (EPM-UL) 26.1.0 or later so that it uses certificates issued and managed by your own organization, instead of certificates generated and managed by EPM-UL.

This information applies to on-premises, new installations.

Use this topic when:

• You are deploying EPM-UL on premises, and
• Your organization already operates an internal PKI, and
• Policy requires TLS certificates to be issued by, or chained to, your CA.

Overview of certificate roles

Roles when EPM-UL uses certificates from your own PKI:

• Server certificate and key (sslservercertfile, sslserverkeyfile) Presented by the EPM-UL daemons — the policy server (pbmasterd), the run host (pblocald), and the log server (pblogd). A daemon uses this certificate whichever end of the connection it is on: when the policy server connects out to a run host or a log server it still presents its server certificate, not a client one. Every host that runs a daemon therefore needs sslservercertfile and sslserverkeyfile, including run hosts.
• REST service certificate and key — not the sslserver* keywords The REST service (pblighttpd) does not read sslservercertfile or sslserverkeyfile. It reads the certificate and key from the paths configured in its own pblighttpd.conf (ssl.pemfile and ssl.privkey). In EPM-UL-managed mode the product copies the server certificate and key to those paths for you; in customer-managed mode you place them there yourself. See Configure the REST service certificate.
• Client (endpoint) certificate and key (sslpbruncertfile, sslpbrunkeyfile) Presented only by the user-invoked client programs on a submit host — pbrun, pbsh, pbksh, and pbssh — both when they contact the policy server and when they write directly to a log server.
• Trusted CA location (sslservercadir / sslservercafile, and sslpbruncadir / sslpbruncafile) A directory of hashed CA certificates and/or a CA bundle file, used to verify peers (default /etc/pbcerts). It must contain your root CA — see below.
• Optional subject pinning (sslserververifysubject, sslpbrunverifysubject) A regular expression matched against the peer certificate's subject distinguished name. This is independent of host-name verification, which is controlled separately by ssloptions (see Host name verification).

Which certificate a host presents

A log server that requires client certificates will therefore see sslpbrun* certificates from submit hosts and sslserver* certificates from policy servers and run hosts. Both must be trusted.

Extended key usage

Issue every EPM-UL leaf certificate with both serverAuth and clientAuth extended key usage.

sslservercertfile requires both, because a daemon presents it at either end of a connection: as the TLS server when it accepts a connection, and as the TLS client when it connects out to a run host or a log server. A certificate with serverAuth alone is rejected by the peer on those outbound connections.

sslpbruncertfile is only ever presented as a TLS client, so clientAuth is what the handshake requires — but issue it with both anyway. EPM-UL classifies a leaf certificate that does not carry both as unusable when its tooling inspects the configured certificates, and a certificate with no extended key usage extension at all is treated the same way.

Because both roles want the same extended key usage, the simplest arrangement is one certificate and key per host, referenced by both keyword families:

sslservercertfile   /etc/pbssl.pem
sslserverkeyfile    /etc/pbsslkey.pem
sslpbruncertfile    /etc/pbssl.pem
sslpbrunkeyfile     /etc/pbsslkey.pem

That is the recommended layout unless you have a specific reason to separate them — it halves the number of certificates to issue, renew and distribute, and it removes a class of mistake where a host's two certificates drift apart.

Install the primary server with your own certificates

You can install from the command line or by using BIUL (Managing Hosts). The same certificate options apply.

To install the primary server:

1. Run pbinstall with -n no. That option tells the installer not to check for or auto-create default certificates and keys, which is what selects customer-managed mode. The rest of the installation is unchanged, so choose components as you normally would.
2. Place your certificate files on the host.
3. Configure the certificate keywords and ssloptions in pb.settings. The certificate keywords say which certificates exist; ssloptions says what EPM-UL does with them.

# where the certificates are
sslservercertfile   /etc/pbssl.pem
sslserverkeyfile    /etc/pbsslkey.pem
sslservercadir      /etc/pbcerts
sslpbruncertfile    /etc/pbssl.pem
sslpbrunkeyfile     /etc/pbsslkey.pem
sslpbruncadir       /etc/pbcerts

# what EPM-UL does with them -- do not omit this line
ssloptions          RequireSSL ValidateClient

A single combined PEM that contains both the certificate and the key is also supported: set both the certificate and key keywords to the same file.

Upgrade a host that uses your certificates

When you upgrade without -n no, the installer inspects the certificates already configured and turns auto-generation off by itself only if every one of them was issued by EPM-UL. Certificates from your own PKI do not qualify, so auto-generation stays enabled and the upgrade continues down the EPM-UL-managed path: an interactive upgrade asks whether you want EPM-UL to install its default certificates, and an unattended one proceeds without asking.

In practice an upgrade that omits -n no usually stops before it can do any harm, because auto-generation needs registration details it does not have:

To install/update the Privilege Management for Unix & Linux default certificates, the following
command line options are required: -A -K -D -P -G
Please re-run pbinstall with the required options or use the '-n no' option to disable
auto-creation of default certificates/keys.

The installation aborts and pb.settings is left untouched. If you do supply those options, auto-generation proceeds and the installer then examines each configured certificate: one it cannot classify — including a valid certificate from your PKI that lacks both serverAuth and clientAuth extended key usage — is reported as an insecure self-signed certificate, and you are offered the chance to replace it with EPM-UL-issued certificates.

Passing -n no avoids both outcomes. Issuing your certificates with both extended key usages (see Extended key usage) removes the underlying risk as well: a certificate that carries both is recognized as externally issued and is never a candidate for replacement.

Configure trusted CA locations

Two separate things have to be in place, and it helps to keep them straight:

• The trust anchor, your root CA, goes in the trusted CA location. This is what EPM-UL verifies against.
• The chain, the intermediate CAs between your root and each leaf, has to reach the peer somehow.

sslservercadir   /etc/pbcerts
sslpbruncadir    /etc/pbcerts

After you place or change CA certificates in the directory, run openssl rehash so the SSL library can locate them by subject hash:

openssl rehash /etc/pbcerts

Delivering the intermediate CAs

Preferred: send the chain with the certificate. Append your intermediate CA certificates, in order from the issuing CA upwards, after the leaf certificate in the file named by sslservercertfile (and sslpbruncertfile, and the REST rest.pem):

-----BEGIN CERTIFICATE-----   <- the host's leaf certificate
...
-----BEGIN CERTIFICATE-----   <- the CA that issued it
...
-----BEGIN CERTIFICATE-----   <- that CA's issuer, and so on up to (but not including) the root
...

EPM-UL loads the whole file and presents every certificate in it during the handshake, so any peer can build the chain whether or not it holds your intermediates locally. Do not append the root — the peer must already have it, and sending it achieves nothing. This is also how EPM-UL handles its own managed certificates: it embeds its sub CA in the certificate file so that every handshake carries it.

Alternative: publish the intermediates locally. You may instead place the intermediates in the trusted CA directory alongside the root and rehash. EPM-UL can then complete a chain for a peer that sends only its leaf.

Use this approach when you cannot control what a peer sends. It is weaker as a general practice, because you have to repeat it on every host that might verify that peer, and it fails silently on any host that was missed. A certificate that carries its own chain works everywhere. Placing an intermediate in the trusted CA directory does not make it a trust anchor: verification still has to reach your root (see above).

You can do both, and on an estate with more than one issuing CA that is often the simplest arrangement.

Configure the REST service certificate

The EPM-UL REST service (pblighttpd) does not use the sslserver* keywords at all. It presents a certificate read from the paths configured in its own pblighttpd.conf, at a fixed location. In EPM-UL-managed mode the product copies the server certificate and key to those paths; in customer-managed mode you supply them. The installer generates <pbrestdir>/etc/pblighttpd.conf from a template on a fresh install and owns its ssl.pemfile / ssl.privkey directives, so treat the REST certificate paths as installer-managed — do not rely on hand-edits to pblighttpd.conf persisting (a fresh reinstall regenerates the file; an upgrade preserves the existing one).

The key layout depends on whether ssl.privkey is active in your pblighttpd.conf:

• Separate (standard managed install — ssl.privkey active): certificate in rest.pem, key in restkey.pem.
• Combined (a fresh -n no install comments out ssl.privkey, so the listener reads the key from rest.pem; this is also the pre-26.1 form): certificate and key together in rest.pem.

Check the ssl.privkey line in pblighttpd.conf, then provide the matching layout as a copy of your server certificate/key, owned by the pblight REST user. For a customer-managed (-n no) install, that is the combined form:

# Resolve <pbrestdir> with:  pbdbutil --cfg -g pbrestdir

# Combined layout (fresh -n no install — ssl.privkey commented):
cat /etc/pbssl.pem /etc/pbsslkey.pem > /tmp/rest.pem
install -o pblight -g pblight -m 0600 /tmp/rest.pem <pbrestdir>/ssl/rest.pem && rm -f /tmp/rest.pem

# Separate layout (ssl.privkey active — e.g. a managed install):
#   install -o pblight -g pblight -m 0644 /etc/pbssl.pem    <pbrestdir>/ssl/rest.pem
#   install -o pblight -g pblight -m 0600 /etc/pbsslkey.pem <pbrestdir>/ssl/restkey.pem

The REST service does not validate certificates

The REST service presents a certificate; it does not validate anybody else's. The shipped pblighttpd.conf configures ssl.engine, ssl.pemfile, ssl.privkey and the cipher and protocol options, and nothing else — there is no ssl.ca-file and no ssl.verifyclient. Consequences worth being explicit about:

• A REST caller is authenticated by its HMAC application key, not by a client certificate. Do not plan a deployment on the assumption that REST enforces mutual TLS.
• The keywords that govern validation elsewhere in EPM-UL — sslservercadir, ssloptions, ClientCertificates, ValidateClient — have no effect on the REST listener.
• Nothing checks the REST certificate's own validity either: pblighttpd will serve an expired or revoked certificate.

pblighttpd is lighttpd, so its own client-certificate directives (ssl.ca-file, ssl.verifyclient.activate, ssl.verifyclient.enforce) can be added to pblighttpd.conf by hand, and they do take effect — a caller without a certificate is then refused at the TLS layer. Two things to understand before you do it.

It does not replace application-key authentication. A caller that presents an acceptable certificate still has to supply its HMAC application key; the certificate only gets it through TLS.

It is unmanaged, and an uninstall silently discards it. The file is generated and maintained by the installer and is not part of the documented configuration surface. Installing over an existing installation leaves your edits alone — but removing and reinstalling the product destroys them, because the uninstaller deletes the REST directory and the installer then writes a fresh pblighttpd.conf from its template. Under -n no that also re-comments ssl.privkey, returning the host to the combined-key layout. Keep any such changes in your own configuration management, and re-apply them after any uninstall/reinstall.

Install additional servers and endpoints

Secondary servers and endpoints join the primary through client registration, using the usual registration options. The root-CA fingerprint is not required under -n no.

About client registration

Client registration fetches a profile from the primary and applies it: the profile can deliver a pb.settings file, shared certificates/keys, and generated certificates. -n no disables only the installer's default certificate creation — it does not govern what the profile delivers. In a customer-managed deployment, either:

• Author the registration profile on the primary to deliver customer-managed settings and no generated certificates, then apply your certificate files and keywords after registration (registration writes pb.settings during install); or
• Skip client registration (--skip-clientreg) and configure each host's settings and certificates manually.

Distributing CA material through the registration profile

A registration profile can carry your CA material to each host, which avoids placing it by hand. Two item types do the work:

• save copies a file to a destination on the client. Give it either fname (a path on the primary) or sname (a settings keyword, whose file is then fetched), plus to (where to write it).
• run fetches a script and executes it on the client as root, so the trusted CA directory can be rehashed after the delivery.

{"name":"customer_ca","profile":[
  {"type":"save","fname":"/etc/pbdist/root.crt","to":"/etc/pbcerts/root.crt"},
  {"type":"run","fname":"/etc/pbdist/rehash.sh"}]}

where rehash.sh is simply:

#!/bin/sh
openssl rehash /etc/pbcerts

Author the profile on the primary with pbdbutil --reg -u, then register clients against it with pbregister -N <profile>. The same mechanism can deliver intermediate CAs for the alternative chain arrangement described in Delivering the intermediate CAs.

For each additional policy or log server, and each endpoint:

1. Run pbinstall with -n no and the usual registration options. Registration runs first and writes pb.settings.
2. After registration completes, install the host's certificate and key and (re)apply the keywords — sslservercertfile / sslserverkeyfile (servers) or sslpbruncertfile / sslpbrunkeyfile (endpoints), plus the trusted CA location — so your customer-managed configuration is the final state, not whatever the profile delivered.
3. Confirm the certificate chain validates against the trusted CA.

Mutual TLS and verification

Verification strictness is controlled by the ssloptions setting:

• RequireSSL: Require TLS; no plaintext fallback.
• ValidateServer: Endpoints validate the server certificate against the trusted CA, and verify the server's host name (see below).
• ClientCertificates: Servers require the endpoint to present a client certificate and validate it against the trusted CA. Every endpoint must then have sslpbruncertfile and sslpbrunkeyfile configured, or the connection is refused.
• ValidateClient: Everything ClientCertificates does, plus host-name verification of the endpoint. Setting ValidateClient implies ClientCertificates; you do not need both.
• AllowLegacySSLCerts: Disables peer certificate verification. See the warning below.

When a trusted CA location is configured and AllowLegacySSLCerts is not set, EPM-UL validates the peer's certificate chain automatically. A customer-managed deployment therefore needs no extra option to get chain validation — only a populated, rehashed CA location.

Recommended settings

Build up in three steps, so that a failure at any stage tells you which control caused it:

ValidateServer and ValidateClient both add host-name verification, so adopt them only once forward and reverse DNS agree with your certificate SANs — see Host name verification. Add SSLFirst to negotiate TLS before the EPM-UL protocol rather than after it; it cannot be combined with AllowNonSSL, and it requires every peer to support it, so introduce it after the rest of the estate is converted.

ValidateServer and ValidateClient need a usable trusted CA location

These options do not check that your CA location contains anything — they simply cause verification to be enforced, and it then fails. What you see depends on how the location is wrong:

EPM-UL rejects non-TLS connections alongside either option is rejected outright, at startup: Invalid ssloptions - AllowNonSSL is not allowed when ValidateClient or ValidateServer is specified. An invalid ssloptions value prevents pbadmin from starting, so you cannot use pbadmin --cfg to undo it — edit ssloptions in pb.settings directly.

Host name verification

By default EPM-UL validates the certificate chain but does not check host names. Setting ValidateServer (checked by the endpoint against the server) or ValidateClient (checked by the server against the endpoint) additionally enables standard host-name verification. Two details matter when you issue the certificates:

• The name checked is the peer's fully qualified domain name as resolved from the connected IP address, not the name typed on the command line. That name must appear in the certificate's Subject Alternative Name extension, so make sure forward and reverse DNS agree.
• Partial wildcards are rejected. *.example.com matches host.example.com; w*.example.com does not match anything. A wildcard also does not match the apex (example.com) or a deeper label (a.b.example.com).

If you cannot guarantee reverse DNS, leave ValidateServer and ValidateClient unset and rely on chain validation, optionally with sslserververifysubject / sslpbrunverifysubject to pin the subject distinguished name instead.

Manage the certificate lifecycle

In customer-managed mode, you own the full certificate lifecycle. EPM-UL does not auto-renew certificates, auto-enroll new hosts, or maintain certificate revocation lists. You must:

• Replace certificates before they expire.
• Provision new hosts with their certificates.
• Distribute revocation information using your own processes.

Troubleshoot certificate issues

First, rule out the two faults that look like certificate problems

Both of these produce the same client-side output as a trust failure — 3018 LIBMANGLE_mangleRecvStart failure on fd 3. No data returned followed by 3003 Could not connect to a Policy Server daemon — and both leave nothing in the policy server's log, because the server exits before it writes one. Check them before you look at certificates.

1. Validate the policy — on the policy server. An uncompilable policy produces two different client-side symptoms, and neither one mentions the policy:

• 3018 LIBMANGLE_mangleRecvStart failure then 3003, because the server exits without logging; or
• Error(s) in config file. Contact your local system administrator. — which points the operator at the client's configuration even though the fault is on the server.

If you see either, check the policy before you change anything locally:

pbcheck -t -f /etc/pb.conf

A clean policy reports "Type check completed with no problems detected."

2. Confirm the network encryption key matches on every host. When networkencryption names a key file — the default is aes-256:keyfile=/etc/pb.key — every host in the installation must hold the same key. Each installation generates its own, so hosts installed separately will not interoperate until you copy one key to all of them:

# on each host
md5sum /etc/pb.key

If the sums differ, copy the policy server's /etc/pb.key to the other hosts (mode 0400, owned by root) and retry. This is unrelated to your PKI: the key protects the EPM-UL protocol inside the TLS session, and a mismatch is not a certificate fault.

Then trace the handshake

If a connection fails, trace the TLS handshake directly from a client:

pbrun -d tls id       # trace the TLS handshake (add a level for more detail, e.g. -d tls=2)
pbrun -d connect id   # trace connection attempts and failover across addresses and servers

A successful handshake ends with HANDSHAKE_DONE SSLOK. A trust or chain problem appears as a fatal alert — for example descr:CA unknown CA followed by verify failed — which means the peer certificate does not chain to a CA in your trusted CA location. Confirm the chain with openssl verify -CAfile <your-ca> <cert>, and that the CA directory is populated and rehashed.

For other failures, review pbmasterd.log and pbrun.log. EPM-UL reports a generic 3430 Insecure operation message to end users; to see the underlying error, set warnuseronerror=yes and showunsecurewarnings=yes, and define the relevant log paths (for example, pbrunlog) to a secure location. For example, a missing client key then reports 8501 Can not stat SSL private key file ....