Visit Azul.com Support

Setting Up the Forwarder

Table of Contents
Need help?
Schedule a consultation with an Azul performance expert.
Contact Us

Setting up a Forwarder involves steps both in the Azul Intelligence Cloud Web UI and on a machine within your environment. This step-by-step guide will help you in this process.

Configuration in the Web UI

Administrators must set up a Forwarder in the Azul Intelligence Cloud Web UI to connect it to their account. Forwarders are created on a per-Azul Intelligence Cloud basis and can be managed by administrators.

Visit the Web UI of your Azul Intelligence Cloud instance.

Adding a Forwarder

  1. In the Setup section of the Web UI click "Forwarder".

  2. In the "Forwarders list", click the "Create Forwarder" button.

  3. Fill in the fields to configure your Forwarder.

    • Forwarder name: this is a free value. It is a name to manage your Forwarder more easily, not a domain name. If your organization uses multiple Forwarders, consider a naming convention that signifies usage.

    • Forwarder’s Domain Names: a list of domain names that your JDKs will use to communicate with this Forwarder. The first entry will be your certificate’s Common Name (CN) and subsequent entries will be Server Alternative Names (SAN).

  4. Click the "Create" button to store the new Forwarder.

  5. The "Forwarder ID" and "Access key" will be shown, together with the pre-populated settings when you click on "Show settings". You can also download them as a ready-to-use properties file with the "Download as file" button.

Note
 For security reasons, by default the Forwarder is not allowed to run as a server in a public domain to avoid unwanted behaviour like sending sensitive JVM data to the public internet and causing incorrect traffic on the domain.

If your environment requires the use of a public domain, you can request Azul support to change the configuration of your Azul Intelligence Cloud environment to add it as an approved domain.

Configure Your Firewall

The Forwarder will need to have access to the Azul Intelligence Cloud and AWS S3 services, through the firewall of your system.

Two rules are needed with outgoing access:

  • The host address (API URL) on port 443 (HTTPS) of the Azul Intelligence Cloud service that has been set up for you by Azul. You can copy this from the Web UI Forwarder setup screen. For instance: YOUR_INSTANCE.api.ic.azul.com.

  • To be able to upload artifacts, outgoing traffic to *.amazonaws.com must also be allowed.

Setting up the Forwarder in Your Environment

The remaining configuration of the Forwarder has to be done within your environment.

Server Requirements

The Forwarder can run on any server within your environment at a location where it can be reached by many JVMs and the Forwarder itself can reach Azul Intelligence Cloud. The only requirement is a Java runtime, version 8 or newer.

Downloading the Forwarder

The Forwarder application can be downloaded from the Azul repository:

 
curl -O https://cdn.azul.com/forwarder/1.0.7/forwarder.jar

Configuring the Forwarder

Three different possibilities are supported to configure the Forwarder, and the values defined in each of them have this priority:

  1. System variables

  2. Properties file

  3. Environment variables

Some settings are the ones filled in or retrieved from the Web UI earlier, others are related to the created certificate. The settings will be used by the Forwarder to connect to Azul Intelligence Cloud. Once authenticated, the Forwarder will exchange this information with Azul Intelligence Cloud to receive cryptographic certificates that will be trusted by the JVM.

This is an overview of the available configuration options:

Environment System Properties

IC_HOST

icHost

ic.host

The host of Azul Intelligence Cloud, copied from the Web UI Forwarder setup screen.

CLOUD_CONNECTION_POOL_SIZE

cloudConnectionPoolSize

cloud.connection.poolSize

Connection pool size used to manage connections between the Forwarder and Azul Intelligence Cloud. The default value is 50.

Example:

 
export CLOUD_CONNECTION_POOL_SIZE=100

CLOUD_RESPONSE_MAX_CONTENT_LENGTH

cloudResponseMaxContentLength

cloud.response.maxContentLength

The maximum length of response content, in bytes, that the Intelligence Cloud can send to the virtual machine. The default value is 524288.

FORWARDER_ACCESS_KEY

forwarderAccessKey

forwarder.access.key

The Forwarder Access Key copied in the Azul Intelligence Cloud Web UI after adding the Forwarder. This Access Key will be used for authentication to Azul Intelligence Cloud.

FORWARDER_DOMAIN_NAMES

forwarderDomainNames

forwarder.domain.names

A comma-delimited list with he Forwarder Domain Names (all or subset of them) that were used to create the Forwarder in the Azul Intelligence Cloud Web UI. These Domain Names will be used as SN/SAN attributes when creating the certificates for the authorization phase between the Forwarder and Azul Intelligence Cloud.

This list can contain IP addresses, but must contain at least one domain name.

FORWARDER_KEYSTORE_PATH

forwarderKeystorePath

forwarder.keystore.path

If no JKS file is found in the specified path, an empty keystore with the same name will be created. This keystore is used to store the auto-generated Forwarder certificates.

FORWARDER_KEYSTORE_PASSWORD

forwarderKeystorePassword

forwarder.keystore.password

FORWARDER_PORT

forwarderPort

forwarder.port

The Forwarder will listen on this port for incoming connections from VMs. The default value is 443.

FORWARDER_STATE_REPORT_PATH

forwarderStateReportPath

forwarder.state.reportPath

File path used by the Forwarder to periodically store its state.

Example:

 
export FORWARDER_STATE_REPORT_PATH=./forwarder-state.log

FORWARDER_STATE_CHECK_PERIOD

forwarderStateCheckPeriod

forwarder.state.checkPeriod

Human-readable period specifying how often the state of the Forwarder should be checked and stored in the report path. It can be defined in the following formats:

  • {N}d

  • {N}h

  • {N}min

  • {N}sec

  • {N}ms

Example:

 
export FORWARDER_STATE_CHECK_PERIOD=10sec

LOG_LEVEL

logLevel

log.level

The default log level is INFO.

By default, when errors are caught in the Forwarder, only error messages will be logged, without a stack trace. If you need to see the stack trace for debugging or troubleshooting purposes, use the TRACE level.

Possible values (case-insensitive):

  • ALL

  • TRACE

  • DEBUG

  • INFO

  • WARN

  • ERROR

  • OFF
Note
 If you are concerned about the security of the password (FORWARDER_KEYSTORE_PASSWORD) and/or key (FORWARDER_ACCESS_KEY) values, using environment variables is the recommended approach.

Using Properties File

When you want to use a properties-file to configure the Forwarder, follow these steps.

  1. Create a new file forwarder-settings.properties

     
    touch forwarder-settings.properties

  2. Edit the file and add the following content, specifying all the required values as defined in the table above.

     
    ic.host = ${intelligence-cloud-api-host}
forwarder.access.key = ${forwarder-access-key}
forwarder.domain.names = <domain-names-from-web-ui>
forwarder.keystore.path = <path-to-keystore>/forwarder.jks
forwarder.keystore.password = <desired keystore password>
forwarder.port = 443

Start the Forwarder Application

Start the Forwarder application with the following command:

 
$JAVA_HOME/bin/java -DforwarderSettings=forwarder-settings.properties -jar forwarder.jar

Checking the State of the Forwarder

Unsuccessful Start

If either the Forwarder Access Key or Common Name does not match, the Forwarder will display an appropriate error message and fail to start. Keep an eye on the output of the Forwarder to check if it could start successfully. These are some possible error messages while connecting:

 
Failed to init certificate challenge: responseCode=403, responseContent={"message":"Forbidden"}
Failed to authorize forwarder, the application will be stopped.

Successful Start

A successful startup will contain these info messages:

 
Forwarder certificate signed by IC issued successfully
Start forwarding *:443 to YOUR_AVD_HOST:443

Status Screen

Also after a successful startup, a status page is available at the address https://{FORWARDER_ENDPOINT}.

Status API

The health of the Forwarder can also be checked via an API on https://{FORWARDER_ENDPOINT}/forwarder/health, which will return, for example:

 
{
  "version":"1.0.7",
  "state":"GREEN",
  "uptime":"327454123",
  "formattedUptime":"03d 18h 57m 34s"
}

Or more extended data can be retrieved by adding ?extended=true like this https://{FORWARDER_ENDPOINT}/forwarder/health?extended=true, resulting in, for example:

 
{
  "state": "RED",
  "version": "1.0.7",
  "uptime": "100610966",
  "formattedUptime": "01d 03h 56m 50s",
  "reasons": [
    "CPU usage exceeded the RED threshold: usage=86%, threshold=85%",
    "Heap memory usage (70%) has exceeded 65%"
  ],
  "memory": {
    "heap.max": "10485760",
    "heap.used": "7361768",
    "heap.committed": "10485760",
    "heap.init": "10485760",
    "nonHeap.max": "-1",
    "nonHeap.used": "28254384",
    "nonHeap.committed": "31522816",
    "nonHeap.init": "7667712"
  },
  "cpu": {
    "jvmCpuUsage": "86%",
    "systemCpuUsage": "97%"
  }
}

Forwarder State

The state of the Forwarder is shown in both the Status-page and the API as GREEN in normal state or one of the following:

Metrics Details Threshold State

OperatingSystemMXBean#ProcessCpuLoad

"recent cpu usage" for the Java Virtual Machine process

50%

YELLOW

OperatingSystemMXBean#ProcessCpuLoad

"recent cpu usage" for the Java Virtual Machine process

85%

RED

MemoryMXBean#HeapMemoryUsage

[amount of used memory in bytes] / [maximum amount of memory in bytes that can be used for memory management]

65%

YELLOW

MemoryMXBean#HeapMemoryUsage

[amount of used memory in bytes] / [maximum amount of memory in bytes that can be used for memory management]

90%

RED