Using the Metric Streaming Message Bus (MSMB)

The Metric Streaming Message Bus (MSMB) is an interface that uses asynchronous messaging to notify subscribers about the most recent metrics of the managed resources. You can configure the interval and the metrics that you want to receive using the REST APIs.

Connect to the MSMB

Prerequisites

To use the MSMB, you must do the following tasks:

  • Use REST APIs to create and download an Advanced Message Queuing Protocol (AMQP) certificate from the appliance.

  • Connect to the MSMB using one or both of these methods:

    • Use the “EXTERNAL” authentication mechanism

    • Connect without sending a user name and password

    Using one of these methods ensures that certificate-based authentication is used.

  • Set up a queue with an empty queue name.

    AMQP generates a unique queue name. You use this queue name to bind to exchanges and receive messages.

Create and download the AMQP client certificate

Creating and downloading the client certificate, private key, and root CA certificate
  1. Create the certificate.

    POST /rest/certificates/client/rabbitmq

    Request body: {"type":"RabbitMqClientCertV2","commonName":"default"}

  2. Download the certificate and private key.

    GET /rest/certificates/client/rabbitmq/keypair/default

  3. Download the root CA certificate.

    GET /rest/certificates/ca

  4. After you connect the client to the MSMB, you can Set up a queue to connect to the HPE OneView MSMB exchange.

Connecting the client to the MSMB

Connecting the client to the MSMB

1

The MSMB consumer requests a client certificate as part of the registration process.

2

The appliance manages the client certificates in a JVK (Java KeyStore) file.

3

The appliance issues a client certificate to the MSMB consumer.

4

The MSMB client provides a SSL client certificate to create a connection with the appliance.

5

The appliance can revoke the MSMB client certificate to deny access to the MSMB client. The client is managed into a CRL (Certificate Revocation List) file.

6

The appliance authenticates the MSMB client using the client certificate.

Set up a queue to connect to the HPE OneView MSMB exchange

The metric streaming messages are published to the HPE OneView MSMB exchange name. To subscribe to messages, you must create a queue or connect to an existing queue that receives messages from the MSMB exchange based on a routing key.

When you create a queue, you define the routing key associated with the queue to receive specific messages.

Exchange Name: msmb

Routing Key: msmb.#

where:

msmb

The HPE OneView exchange name for metric streaming.

Sample queues

Subscription Example
Receive all MSMB messages for physical servers, enclosures, and power devices The exchange is msmb

The routing key is msmb.#

Configure metric relay using Metric Streaming configuration API.

JSON structure of message received from the MSMB

The following table lists the attributes included in the JSON payload of each message from the MSMB. The resource model for the HPE OneView resource is included in the resource attribute. To view all resource models, see the HPE OneView REST API Reference.

Attribute Data type Description
resourceUri String The URI for the resource.
changeType String The state-change type: Created, Updated, or Deleted.
newState String The new state of the resource.
eTag String The ETag for the resource when the state change occurred.
timestamp String The time the message was sent.
newSubState String If substate messages are required (for substate machines associated with a primary state), this is the resource-specific substate.
resource MetricData The resource model.
associatedTask String If a task is not associated with this message, the value is null.
userInitiatedTask String The value of the userInitiated attribute included in the associatedTask attribute.
changedAttributes Array A list of top-level attributes that have changed based on the POST or PUT call that caused the state-change message to be sent.
data Object Additional information about the resource state change.

MetricData

Attribute Data type Description
startTime String The starting time of the metric collection.
sampleIntervalInSeconds Integer Interval between samples.
numberOfSamples Integer Number of samples in the list for each metric type.
resourceType String Identifies the category of resource. The supported devices are server-hardware, enclosures, and power-devices.
resourceDataList List Metric sample list.
uri String Canonical URI of the resource.
category String Identifies the category of resource. The supported devices are server-hardware, enclosures, and power-devices.
created Timestamp Date and time when the resource was created.
modified Timestamp Date and time when the resource was last modified.
eTag String Entity tag/version ID of the resource, the same value that is returned in the ETag header on a GET of the resource.
type String Uniquely identifies the type of the JSON object.

Structure of message received from the MSMB

{
    "eTag": null,
    "resourceUri": "/rest/enclosures/09SGH100X6J1",
    "changeType": "Updated",
    "newState": null,
    "newSubState": null,
    "associatedTask": null,
    "userInitiatedTask": false,
    "changedAttributes": null,
    "data": null,
    "resource": {
        "type": "MetricData",
        "resourceType": "enclosures",
        "resourceDataList": [
            {
                "metricSampleList": [
                    {
                        "valueArray": [
                            null
                        ],
                        "name": "RatedCapacity"
                    },
                    {
                        "valueArray": [
                            523
                        ],
                        "name": "AveragePower"
                    },
                    {
                        "valueArray": [
                            573
                        ],
                        "name": "PeakPower"
                    },
                    {
                        "valueArray": [
                            null
                        ],
                        "name": "PowerCap"
                    },
                    {
                        "valueArray": [
                            23
                        ],
                        "name": "AmbientTemperature"
                    },
                    {
                        "valueArray": [
                            null
                        ],
                        "name": "DeratedCapacity"
                    }
                ],
                "resourceId": "09SGH100X6J1"
            }
        ],
        "numberOfSamples": 1,
        "sampleIntervalInSeconds": 300,
        "startTime": "2014-09-17T08:43:36.294Z",
        "eTag": null,
        "modified": null,
        "created": null,
        "category": "enclosures",
        "uri": "/rest/enclosures/09SGH100X6J1"
    },
    "timestamp": "2014-09-17T08:48:36.819Z"
}

Example to connect and subscribe to MSMB using .NET C#

Prerequisites

In addition to completing the prerequisites, you must complete the example-specific prerequisites before using the .NET C# examples.

To use the .Net C# examples, add the following to the Windows certificate store:

  • CA root certificate.

  • Client certificate

  • Private key

To try the .Net C# examples, do the following:

  1. Download the root CA certificate.

    GET /rest/certificates/ca

  2. Save the contents in the response body into a text file named rootCA.crt. You must copy and paste everything from -----BEGIN CERTIFICATE----- to -----END CERTIFICATE-----, including the dashes, but not including the quotes.

  3. Import the rootCA.crt file into the Windows certificate store under Trusted Root Certification Authorities.

  4. Download the client certificate and private key.

    GET /rest/certificates/client/rabbitmq/keypair/default

  5. Save the contents of the client certificate and private key in the response body into a text file named msmb.crt.

    You must copy and paste everything from -----BEGIN CERTIFICATE----- to -----END CERTIFICATE----- for the client certificate. Next, copy and paste everything from -----BEGIN RSA PRIVATE KEY----- to -----END RSA PRIVATE KEY----- for the private key. You must include the dashes, but do not include the quotes.

Using .Net C# to directly reference client certificate

Convert the client certificate and private key to PKCS format for .Net.

openssl.exe pkcs12 -passout pass:default -export -in msmb.crt -out msmb.p12

Example 

  public void Connect()
        {
            string exchangeName = "msmb";
            string hostName = "OneView.domain";
            string queueName = "";
            string routingKey = "msmb.#";

            ConnectionFactory factory = new ConnectionFactory();
            factory.AuthMechanisms = new RabbitMQ.Client.AuthMechanismFactory[] { new ExternalMechanismFactory() };
            factory.HostName = hostname;
            factory.Port = 5671;
            factory.Ssl.CertPath = @".\msmb.p12";
            factory.Ssl.CertPassphrase = "default";
            factory.Ssl.ServerName = hostname;
            factory.Ssl.Enabled = true;

            IConnection connection = factory.CreateConnection();
            IModel model = connection.CreateModel();

            queueName = model.QueueDeclare(queueName, false, false, false, null);
            model.QueueBind(queueName, exchangeName, routingKey, null);

            using (Subscription sub = new Subscription(model, queueName))
            {
                foreach (BasicDeliverEventArgs ev in sub)
                { 
                    DoSomethingWithMessage(ev);
                    sub.Ack();
                }
            }
        }

Using .Net C# to import certificate to Microsoft Windows certificate store

Import the msmb.crt into your preferred Windows certificate store.

Example 

public void Connect()
        {
            string exchangeName = "msmb";
            string hostName = "OneView.domain";
            string queueName = "";
            string routingKey = "msmb.#";
            string userName = "rabbitmq_readonly";

            X509Store store = new X509Store(StoreName.Root, StoreLocation.LocalMachine);
            store.Open(OpenFlags.ReadWrite);

            X509Certificate cert = store.Certificates
                 .Find(X509FindType.FindBySubjectName, userName, false)
                 .OfType<X509Certificate>()
                 .First();

            ConnectionFactory factory = new ConnectionFactory();
            factory.AuthMechanisms = new RabbitMQ.Client.AuthMechanismFactory[] { new ExternalMechanismFactory() };
            factory.HostName = hostname;
            factory.Port = 5671;
            factory.Ssl.Certs = new X509CertificateCollection(new X509Certificate[] { cert });
            factory.Ssl.ServerName = hostname;
            factory.Ssl.Enabled = true;

            IConnection connection = factory.CreateConnection();
            IModel model = connection.CreateModel();

            queueName = model.QueueDeclare(queueName, false, false, false, null);
            model.QueueBind(queueName, exchangeName, routingKey, null);

            using (Subscription sub = new Subscription(model, queueName))
            {
                foreach (BasicDeliverEventArgs ev in sub)
                { 
                    DoSomethingWithMessage(ev);
                    sub.Ack();
                }
            }
        }

[NOTE: ]

NOTE: Using .Net C# to import certificate to Microsoft Windows certificate store references the Trusted Root Certificate Authorities store, located under Local Computer.

  • StoreName.Root = Trusted Root Certificate Authorities

  • StortLocation.LocalMachine = Local Computer


Example to connect and subscribe to MSMB using Java

  1. Download the client certificate and private key.

    GET /rest/certificates/client/rabbitmq/keypair/default

  2. Save the contents of the client certificate in the response body into a text file named default-client.crt.

    You must copy and paste everything from -----BEGIN CERTIFICATE----- to -----END CERTIFICATE-----, including the dashes, but not including the quotes.

  3. Save the contents of the private key in the response body into a text file named default-client.key.

    You must copy and paste everything from -----BEGIN RSA PRIVATE KEY----- to -----END RSA PRIVATE KEY-----, including the dashes, but not including the quotes.

  4. Create a PKCS12 keystore from the private key and the public certificate.

    openssl pkcs12 -export -name myclientcert -in default-client.crt -inkey default-client.key -out myclient.p12
    
  5. Convert the PKCS12 keystore into a JKS keystore.

    keytool -importkeystore -destkeystore c:\\MyKeyStore -srckeystore myclient.p12 -srcstoretype pkcs12 -alias myclient
    

Example to connect and subscribe to MSMB using Java

//c://MyKeyStore contains client certificate and private key. Load it into Java Keystore
final char[] keyPassphrase = "MyKeyStorePassword".toCharArray();
final KeyStore ks = KeyStore.getInstance("jks");
ks.load(new FileInputStream("c://MyKeyStore"), keyPassphrase);
final KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509");
kmf.init(ks, keyPassphrase);
 
//c://MyTrustStore contains CA certificate. Load it into Java Trust Store
final char[] trustPassphrase = "MyTrustStorePassword".toCharArray();
final KeyStore ks = KeyStore.getInstance("jks");
tks.load(new FileInputStream("c:\\MyTrustStore"), trustPassphrase);
final TrustManagerFactory tmf = TrustManagerFactory.getInstance("SunX509");
tmf.init(tks);
 
 
//load SSLContext with keystore and truststore.
final SSLContext c = SSLContext.getInstance("SSL");
c.init(kmf.getKeyManagers(), tmf.getTrustManagers(), new SecureRandom());
 
final ConnectionFactory factory = new ConnectionFactory();
factory.setHost("192.168.2.144");
 
//Set Auth mechanism to "EXTERNAL" so that commonName of the client certificate is mapped to AMQP user name. Hence, No need to set userId/Password here. 
factory.setSaslConfig(DefaultSaslConfig.EXTERNAL);
factory.setPort(5671);
factory.useSslProtocol(c);
 
final Connection conn = factory.newConnection();
final Channel channel = conn.createChannel();

//do not specify queue name. AMQP will create a queue with random name starting with amq.gen* e.g. amq.gen-32sfQz95QJ85K_lMBhU6HA
final DeclareOk queue = channel.queueDeclare("", true, false, true, null);
 
//Now get the queue name from above call and bind it to required Exchange with required routing key.
channel.queueBind(queue.getQueue(), "msmb", "msmb.#");
 
//Now you should be able to receive messages from queue
final GetResponse chResponse = channel.basicGet(queue.getQueue(), false);
if (chResponse == null)
{
    System.out.println("No message retrieved");
}
else
{
    final byte[] body = chResponse.getBody();
    System.out.println("Received: " + new String(body));
}
 
channel.close();
conn.close();

Examples to connect and subscribe to MSMB using Python

The Python examples show how to connect and subscribe to the MSMB. For more information about Python (Pika AMQP client library and AMQP client library), see Introduction to Pika (http://pika.readthedocs.org/, http://www.python.org/), and AMQP Client Library (https://pypi.python.org/pypi/amqplib/).

Installation

  1. Install the pika and amqp libraries.

    1. Download and install the setup tools (Python setup.py install) at https://pypi.python.org/pypi/setuptools#downloads.

    2. Install the pika tools.

      When you install the pika or amqp libraries, run the same python setup.py install command from the downloaded pika or amqp directory.

  2. Create the certificate.

    POST /rest/certificates/client/rabbitmq

    Request body: {"type":"RabbitMqClientCertV2","commonName":"default"}

  3. Download the client certificate and private key.

    GET /rest/certificates/client/rabbitmq/keypair/default

  4. Save the contents of the client certificate in the response body into a text file named client.pem.

    You must copy and paste everything from -----BEGIN CERTIFICATE----- to -----END CERTIFICATE-----, including the dashes, but not including the quotes. You must replace all instances of \n with CR/LF (carriage return / line feed).

  5. Save the contents of the private key in the response body into a text file named key.pem.

    You must copy and paste everything from -----BEGIN RSA PRIVATE KEY----- to -----END RSA PRIVATE KEY-----, including the dashes, but not including the quotes. You must replace all instances of \n with CR/LF (carriage return / line feed).

  6. Download the root CA certificate.

    GET /rest/certificates/ca

  7. Save the contents in the response body into a text file named caroot.pem. You must copy and paste everything from -----BEGIN CERTIFICATE----- to -----END CERTIFICATE-----, including the dashes, but not including the quotes. You must replace all instances of \n with CR/LF (carriage return / line feed).

Pika

Pika example

When you invoke the script, you must pass –host:{hostname or IP}. See the following examples:

  • --host:192.168.1.1

  • –host:my-appliance.example.com


[IMPORTANT: ]

IMPORTANT: If the connection fails on the first attempt to invoke this script after an appliance reboot, try invoking the script again.


import pika, ssl
from optparse import OptionParser
from pika.credentials import ExternalCredentials
import json
import logging

logging.basicConfig()

###############################################
# Callback function that handles messages
def callback(ch, method, properties, body):
    msg = json.loads(body)
    timestamp = msg['timestamp']
    resourceUri = msg['resourceUri']
    resource = msg['resource']
    changeType = msg['changeType']

    print
    print ("%s: Message received:" %(timestamp))
    print ("Routing Key: %s" %(method.routing_key))
    print ("Change Type: %s" %(changeType))
    print ("Resource URI: %s" %(resourceUri))
    print ("Resource: %s" %(resource))

#   Pem Files needed, be sure to replace the \n returned from the APIs with CR/LF
#   caroot.pem - the CA Root certificate   - GET /rest/certificates/ca
#   client.pem, first POST /rest/certificates/client/rabbitmq Request body: {"type":"RabbitMqClientCert","commonName":"default"}
#   GET /rest/certificates/client/rabbitmq/keypair/default
#   client.pem is the key with  -----BEGIN CERTIFICATE-----
#   key.pem is the key with -----BEGIN RSA PRIVATE KEY-----
# Setup our ssl options
ssl_options = ({"ca_certs": "caroot.pem",
               "certfile": "client.pem",
               "keyfile": "key.pem",
               "cert_reqs": ssl.CERT_REQUIRED,
               "server_side": False})

parser = OptionParser()
parser.add_option('--host', dest='host',
        help='Pika server to connect to (default: %default)',
        default='localhost',
)

options, args = parser.parse_args()

# Connect to RabbitMQ
host = options.host
print ("Connecting to %s:5671, to change use --host hostName " %(host))
connection = pika.BlockingConnection(
               pika.ConnectionParameters(
                       host, 5671, credentials=ExternalCredentials(),
                       ssl=True, ssl_options=ssl_options))
 
# Create and bind to queue
EXCHANGE_NAME = "msmb"
ROUTING_KEY = "msmb.#"

channel = connection.channel()
result = channel.queue_declare()
queue_name = result.method.queue

channel.queue_bind(exchange=EXCHANGE_NAME, queue=queue_name, routing_key=ROUTING_KEY)

channel.basic_consume(callback,
                      queue=queue_name,
                      no_ack=True)

# Start listening for messages
channel.start_consuming()

AMQP

AMQP example

When you invoke the script, you must pass –host:{hostname or IP}. See the following examples:

  • --host:192.168.1.1

  • –host:my-appliance.example.com


[IMPORTANT: ]

IMPORTANT: If the connection fails on the first attempt to invoke this script after an appliance reboot, try invoking the script again.


#!/usr/bin/env python

from optparse import OptionParser
from functools import partial

import amqplib.client_0_8 as amqp


def callback(channel, msg):
    for key, val in msg.properties.items():
        print ('%s: %s' % (key, str(val)))
    for key, val in msg.delivery_info.items():
        print ('> %s: %s' % (key, str(val)))

    print ('')
    print (msg.body)
    print ('-------')
    print msg.delivery_tag
    channel.basic_ack(msg.delivery_tag)

    #
    # Cancel this callback
    #
    if msg.body == 'quit':
        channel.basic_cancel(msg.consumer_tag)


def main():
    parser = OptionParser()
    parser.add_option('--host', dest='host',
        help='AMQP server to connect to (default: %default)',
        default='localhost',
    )

    options, args = parser.parse_args()
    host = options.host+":5671"
#   Pem Files needed, be sure to replace the \n returned from the APIs with CR/LF
#   caroot.pem - the CA Root certificate   - GET /rest/certificates/ca
#   client.pem, first POST /rest/certificates/client/rabbitmq Request body: {"type":"RabbitMqClientCert","commonName":"default"}
#   GET /rest/certificates/client/rabbitmq/keypair/default
#   client.pem is the key with -----BEGIN CERTIFICATE-----
#   key.pem is the key with -----BEGIN RSA PRIVATE KEY-----

    ssl_options = ({"ca_certs": "caroot.pem",
               "certfile": "client.pem",
               "keyfile": "key.pem",
#               "cert_reqs": CERT_REQUIRED,
               "server_side": False})

    print ('Connecting to host %s, to change use --host hostName ' %host)

    conn = amqp.Connection(host, login_method='EXTERNAL',
                           ssl=ssl_options)

    print ('Successfully connected, creating and binding to queue')

    ch = conn.channel()

    qname, _, _ = ch.queue_declare()
    ch.queue_bind(qname, 'msmb', 'msmb.#')
    ch.basic_consume(qname, callback=partial(callback, ch))

    print ('Successfully bound to queue, waiting for messages')

    #pyamqp://

    #
    # Loop as long as the channel has callbacks registered
    #
    while ch.callbacks:
        ch.wait()

    ch.close()
    conn.close()

if __name__ == '__main__':
    main()

Re-create the AMQP client certificate

If you change the appliance name, you must re-create the AMQP client certificate.


[NOTE: ]

NOTE: If the certificates are already created, you can skip this step.


Prerequisites

  • Minimum required session ID privileges: Infrastructure administrator

Re-creating and downloading the client certificate, private key, and root CA certificate
  1. Revoke the certificate.

    DELETE /rest/certificates/ca/rabbitmq_readonly

    Request body is not required.


    [NOTE: ]

    NOTE: When you revoke the default client certificate, the appliance re-generates the CA certificate, AMQP server certificate, and the default client certificate.


  2. Download the certificate and private key.

    GET /rest/certificates/client/rabbitmq/keypair/default

  3. Download the root CA certificate.

    GET /rest/certificates/ca