Monitoring Optimizer Hub

Of the currently connected client JVMs, how many are running which applications?

Use /api/currentlyConnectedProfileNames to retrieve a list of currently connected profile names and check the value of currentlyConnectedVMInstanceCount.

An example: you can use this approach to find out which two clients are stuck in a reboot loop, causing Cloud Native Compiler to not scale down.

Which application groups are using ReadyNow Orchestrator and which ones are using Cloud Native Compiler?

Call /api/profileNames recursively to get the full list of profile names in an Optimizer Hub instance’s memory then sort by "cncActive": false, "rnoActive": false

How much compute power is each application using?

You may need to charge back the cost for your Optimizer Hub environment to the individual teams, each of which may consume a different amount of resources. Since Cloud Native Compiler is the major consumer of resources, you only need to report per-profile name resource consumption for Cloud Native Compiler. Below you can find two example scripts to achieve this.

You must run this first script periodically (e.g., once a minute) to scrape data from /api/currentlyConnectedProfileNames, and save the result to separate JSON files.

 
import requests
import json

api_base_url = 'http://<your-instance>:8120'

def get_currently_connected_profiles(page_token = None):
    params = {'pageToken': page_token} if page_token is not None else {}
    response = requests.get(f'{api_base_url}/api/currentlyConnectedProfileNames', params)
    response.raise_for_status()
    return response.json()

all_profiles = []
page_token = None

while True:
    response = get_currently_connected_profiles(page_token)
    all_profiles += response['data']
    page_token = response['nextPage']
    if page_token is None:
        break

print(json.dumps(all_profiles, indent = 2))

Once enough JSON files are generated, you can aggregate them (important: you need to do this in the order they were taken) to produce a table of resource usage per profile name for the period of time covered by those scrapes.

 
import sys
import json
from dataclasses import dataclass
from typing import Dict

@dataclass
class Accumulator:
    prev_sample: float = 0
    total: float = 0
    def add(self, sample: float) -> None:
        if sample >= self.prev_sample:
            self.total += sample - self.prev_sample
        else:
            self.total += sample
        self.prev_sample = sample

def read_scrape(path: str) -> Dict[str, float]:
    profile_usage: Dict[str, float] = {}
    with open(path, 'r') as f:
        data = json.load(f)
    for profile in data:
        name = profile['name']
        usage = (profile['cnc'] or {}).get('resouceUsage', 0)
        profile_usage[name] = usage
    return profile_usage

def main(scrape_paths) -> None:
    usage_by_profile: Dict[str, Accumulator] = {}
    for path in scrape_paths:
        scrape = read_scrape(path)
        for name, usage in scrape.items():
            usage_by_profile.setdefault(name, Accumulator())
            usage_by_profile[name].add(usage)

    total_usage = sum(acc.total for acc in usage_by_profile.values())

    for name, acc in sorted(usage_by_profile.items(), key=lambda item: item[1].total, reverse=True):
        percentage = 100 * acc.total / total_usage if total_usage else 0
        print(f'{name:<30} {acc.total:<15.0f} {percentage:>6.2f}%')

if __name__ == "__main__":
    main(sys.argv[1:])

This will list the detected profiles with a percentage of use:

 
exampleProfileName1       7500048012       81.08%
exampleProfileName2       1750024006       18.92%

You can apply these percentages to the total amount your organization has spent on the hosting infrastructure for Optimizer Hub for the same period to split the costs between the applications.

Find out host and port where Loki is listening

 
export LOKI_ADDR=http://{ip-adress}:{port}

Get logs of all pods in the selected namespace

 
logcli query --since 24h --forward --limit=10000 '{namespace="zvm-dev-3606"}'

Get logs of a single application in the selected namespace

 
logcli query --since 24h --forward --limit=10000 '{namespace="zvm-dev-3606" app="compile-broker"}'

Get logs of a single pod in the selected namespace

 
logcli query --since 24h --forward --limit=10000 '{namespace="zvm-dev-3606",pod="compile-broker-5fd956f44f-d5hb2"}'