Automation Blog

from Stefan Schnell


Automated testing of VCF Orchestrator actions opens up powerful possibilities for functional verification and platform diagnosis after new releases. Following a product upgrade, the testing focus is on regression tests and platform validation. Here, actions serve a dual purpose: They act as test objects to verify isolated runtime behavior, and as an active diagnostic tool to analyze the state of internal Orchestrator components. While regression testing ensures that actions deliver their expected outcomes, platform introspection uses these same actions to audit the underlying orchestration platform. The approach presented here makes it possible to execute code via actions in the Orchestrator to implement this dual-purpose.

Automated Platform Introspection via Actions


With the presented approach, the action transforms from a simple code snippet into a gateway for automated platform checks during a release change. The action serves as the central programmable gateway through which the new platform fully and automatically verifies its own operational readiness. On the one hand, code is tested; on the other hand, the same technology is used simultaneously to validate the integrity of the updated platform. The action becomes an active testing tool that audits the platform’s status. It is no longer the object being tested, but rather the component that validates and documents the status of the target components.

The following code is designed in Python to enable execution outside of the VMware Orchestrator environment. The Python script relies strictly on standard libraries and no external dependencies are needed, this ensuring maximum portability. Each Python function can be isolated individually and used without modification in alternative runtime environments or contexts. The lifecycle is fully automated: An action is dynamically built, executed, the result and log are retrieved, and the action is deleted upon completion. On this way a complete test cycle is implemented for both, the Orchestrator actions itself and to test the platform.

Hint: This approach is strictly designed for Orchestrator product testing. The actions under test are not intended to require input parameters, as any necessary configurations can be hardcoded and passed directly within the code. However, support for dynamic parameter passing can be added if required in future extensions.

Hint: To use this approach, you must have the Orchestrator Administrator or Orchestrator Workflow Designer role.

"""
Tests an action via Orchestrator REST API

@author Stefan Schnell <mail@stefan-schnell.de>
@license MIT
@version 0.1.0

@param {string} in_actionModule - Name of the module
@param {string} in_actionName - Name of the action
@param {string} in_outputType - Data type of output, default Properties
@param {string} in_script - Script code
@param {string} in_version - Version of the action, default 0.1.0
@param {string} in_runtime - Name of the runtime, if null JavaScript
@returns {Properties}

@example
System.getModule("de.stschnell").testAction(
  "de.stschnell",
  "helloWorld",
  "string",
  "System.log(\"Hello World\");\n" +
  "return \"Hello World\"",
  null,
  null
);

@example
System.getModule("de.stschnell").testAction(
  "de.stschnell",
  "helloWorld",
  null,
  "import json\n" +
  "def handler(context, inputs):\n" +
  "  print(\"Hello World\")\n" +
  "  return {\"result\": \"Hello World\"}",
  null,
  "python:3.10"
);

Checked with Aria Automation 8.18.1
"""


import ast
import json
import ssl
import time
import urllib.error
import urllib.request


def createAction(
  vcoUrl: str,
  bearerToken: str,
  actionModule: str,
  actionName: str,
  outputType: str | None,
  script: str,
  version: str | None,
  runtime: str | None
) -> dict:
    """ Creates an action in the Orchestrator
    """

    try:

        if not (vcoUrl and vcoUrl.strip()) or \
           not (bearerToken and bearerToken.strip()):
            raise ValueError("vcoUrl or bearerToken argument "
                             "cannot be undefined or null")

        if not (actionModule and actionModule.strip()) or \
           not (actionName and actionName.strip()):
            raise ValueError("actionModule or actionName argument "
                             "cannot be undefined or null")

        _outputType: str = outputType \
        if (outputType and outputType.strip()) else "Properties"

        if not script or not script.strip():
            raise ValueError("script argument "
                             "cannot be undefined or null")

        _version: str = version \
        if (version and version.strip()) else "0.1.0"

        url = f"{vcoUrl.rstrip('/')}/api/actions"

        body: dict = {
            "input-parameters": [],
            "module": actionModule,
            "name": actionName,
            "output-type": _outputType,
            "script": script,
            "version": _version
        }

        if runtime and runtime.strip():
            body["runtime"] = runtime

        request = urllib.request.Request(
            url = url,
            method = "POST",
            data = json.dumps(body).encode("utf-8")
        )

        request.add_header("Content-Type", "application/json")
        request.add_header("Accept", "application/json, text/plain, */*")
        request.add_header("Authorization", f"Bearer {bearerToken}")

        response = urllib.request.urlopen(
            request,
            context = ssl._create_unverified_context()
        )

        responseCode: int = response.status

        if responseCode == 201:
            try:
                return json.loads(response.read().decode("utf-8"))
            except json.JSONDecodeError as jsonErr:
                raise RuntimeError(f"Invalid JSON response: {jsonErr}")
        else:
            raise RuntimeError(f"Unexpected HTTP status code: {responseCode}")

    except urllib.error.HTTPError as err:
        errorDetails = err.read().decode("utf-8")
        raise RuntimeError(f"HTTP error {err.code} at createAction: "
                           f"{errorDetails}") from err
    except Exception as err:
        raise RuntimeError(f"Error at createAction: {err}") from err


def executeAction(
  vcoUrl: str,
  bearerToken: str,
  actionId: str
) -> dict:
    """ Invokes an action in the Orchestrator
    """

    try:

        if not (vcoUrl and vcoUrl.strip()) or \
           not (bearerToken and bearerToken.strip()):
            raise ValueError("vcoUrl or bearerToken argument "
                             "cannot be undefined or null")

        if not actionId or not actionId.strip():
            raise ValueError("actionId argument "
                             "cannot be undefined or null")

        url = f"{vcoUrl.rstrip('/')}/api/actions/{actionId}/executions"

        body: dict = {
            "async-execution": False,
            "parameters": []
        }
        data = bytes(json.dumps(body).encode("utf-8"))

        request = urllib.request.Request(
            url = url,
            method = "POST",
            data = data
        )

        request.add_header("Content-Type", "application/json")
        request.add_header("Accept", "application/json, text/plain, */*")
        request.add_header("Authorization", f"Bearer {bearerToken}")

        response = urllib.request.urlopen(
            request,
            context = ssl._create_unverified_context()
        )

        responseCode: int = response.status

        if responseCode == 200:
            try:
                return json.loads(response.read().decode("utf-8"))
            except json.JSONDecodeError as jsonErr:
                raise RuntimeError(f"Invalid JSON response: {jsonErr}")
        else:
            raise RuntimeError(f"Unexpected HTTP status code: {responseCode}")

    except urllib.error.HTTPError as err:
        errorDetails = err.read().decode("utf-8")
        raise RuntimeError(f"HTTP Error {err.code} at executeAction: "
                           f"{errorDetails}") from err
    except Exception as err:
        raise RuntimeError(f"Error at executeAction: {err}") from err


def getActionLog(
  vcoUrl: str,
  bearerToken: str,
  executionId: str
) -> dict:
    """ Delivers the content of the action log
    """

    try:

        if not (vcoUrl and vcoUrl.strip()) or \
           not (bearerToken and bearerToken.strip()):
            raise ValueError("vcoUrl or bearerToken argument "
                             "cannot be undefined or null")

        if not executionId or not executionId.strip():
            raise ValueError("executionId argument "
                             "cannot be undefined or null")

        url = (f"{vcoUrl.rstrip('/')}/api/actions/{executionId}"
                "/logs?maxResult=2147483647")

        request = urllib.request.Request(
            url = url,
            method = "GET"
        )

        request.add_header("Accept", "application/json, text/plain, */*")
        request.add_header("Authorization", f"Bearer {bearerToken}")

        response = urllib.request.urlopen(
            request,
            context = ssl._create_unverified_context()
        )

        responseCode: int = response.status

        if responseCode == 200:
            rawData = response.read().decode("utf-8")
            try:
                return json.loads(rawData)
            except json.JSONDecodeError as jsonErr:
                try:
                    return ast.literal_eval(rawData)
                except (ValueError, SyntaxError) as astErr:
                    return {"logRawData": rawData}
        else:
            return {}

    except urllib.error.HTTPError as err:
        errorDetails = err.read().decode("utf-8")
        raise RuntimeError(f"HTTP Error {err.code} at getActionLog: "
                           f"{errorDetails}") from err
    except Exception as err:
        print(f"Error at getActionLog: {err}")
        return {}


def deleteAction(
  vcoUrl: str,
  bearerToken: str,
  actionId: str
) -> bool:
    """ Removes an action in the Orchestrator
    """

    try:

        if not (vcoUrl and vcoUrl.strip()) or \
           not (bearerToken and bearerToken.strip()):
            raise ValueError("vcoUrl or bearerToken argument "
                             "cannot be undefined or null")

        if not actionId or not actionId.strip():
            raise ValueError("actionId argument "
                             "cannot be undefined or null")

        url = f"{vcoUrl.rstrip('/')}/api/actions/{actionId}?force=false"

        request = urllib.request.Request(
            url = url,
            method = "DELETE"
        )

        request.add_header("Accept", "application/json, text/plain, */*")
        request.add_header("Authorization", f"Bearer {bearerToken}")

        response = urllib.request.urlopen(
            request,
            context = ssl._create_unverified_context()
        )

        responseCode: int = response.status

        if responseCode == 200:
            return True
        else:
            return False

    except Exception as err:
        print(f"Error at deleteAction: {err}")
        return False


def handler(
    context: dict,
    inputs: dict
) -> dict:

    outputs: dict = {}
    actionId: str | None = None
    executionId: str | None = None

    try:

        vcoUrl: str = context["vcoUrl"]
        bearerToken: str = context["getToken"]()
        actionModule: str = inputs["in_actionModule"]
        actionName: str = inputs["in_actionName"]
        outputType: str | None = inputs["in_outputType"]
        script: str = inputs["in_script"]
        version: str | None = inputs["in_version"]
        runtime: str | None = inputs["in_runtime"]

        action: dict = createAction(
            vcoUrl,
            bearerToken,
            actionModule,
            actionName,
            outputType,
            script,
            version,
            runtime
        )

        actionId = action["id"] if isinstance(action, dict) else None
        if not actionId:
            raise RuntimeError("Action creation returned no valid ID")

        result: dict = executeAction(
            vcoUrl,
            bearerToken,
            actionId
        )

        executionId = result["execution-id"] \
        if isinstance(result, dict) else None
        if not executionId:
            raise RuntimeError("Action execution returned no valid ID")

        log: dict = {}
        maxAttempts = 10 # Increase for long-running or complex tests
        checkInterval = 1
        attemptCounter = 0

        while attemptCounter < maxAttempts:

            log = getActionLog(
                vcoUrl,
                bearerToken,
                executionId
            )

            if log and len(log["logs"]) > 0:
                break

            time.sleep(checkInterval)
            attemptCounter += 1

        if attemptCounter >= maxAttempts:
            print("Warning: Logging timeout, "
                  "logs might be incomplete or missing")

        outputs["status"] = "done"
        outputs["error"] = None
        outputs["createAction"] = action
        outputs["executeAction"] = result
        outputs["logAction"] = log

    except Exception as err:

        outputs["status"] = "incomplete"
        outputs["error"] = str(err)
        outputs["createAction"] = None
        outputs["executeAction"] = None
        outputs["logAction"] = None

    finally:

        if actionId:
            deleted: bool = deleteAction(
              vcoUrl,
              bearerToken,
              actionId
            )
            outputs["deleteAction"] = deleted
        else:
            outputs["deleteAction"] = None

    return outputs


if __name__ == "__main__":
    # Call the sequence from here, if you are not in the Orchestrator
    # createAction()
    # executeAction()
    # getActionLog()
    # deleteAction()
    pass

The following JavaScript code shows a test sequence that tests polyglot code - in this case, JavaScript using the Rhino engine, Python, and PowerShell.

/**
 * Checked with Aria Automation 8.18.1
 */

// Test JavaScript with the Rhino Engine

var jsCode = "\n\
System.log(\"Hello World\");\n\
return \"Hello World\";\n\
";

const jsTest = System.getModule("de.stschnell").testAction(
  "de.stschnell",
  "helloWorld",
  "string",
  jsCode,
  null,
  null
);

System.log(JSON.stringify(jsTest));

// Test Python

const pyCode = "\n\
import json\n\
def handler(context: dict, inputs: dict) -> dict:\n\
    print(\"Hello World\")\n\
    return {\"result\": \"Hello World\"}\n\
"

const pyTest = System.getModule("de.stschnell").testAction(
  "de.stschnell",
  "helloWorld",
  null,
  pyCode,
  null,
  "python:3.10"
);

System.log(JSON.stringify(pyTest));

// Test PowerShell

const psCode = "\n\
function Handler($context, $inputs) {\n\
    Write-Host \"Hello World\"\n\
    $output = @{\n\
        status = \"done\"\n\
        result = \"Hello World\"\n\
    }\n\
    return $output\n\
}\n\
";

const psTest =  System.getModule("de.stschnell").testAction(
  "de.stschnell",
  "helloWorld",
  null,
  psCode,
  null,
  "powershell:7.4"
);

System.log(JSON.stringify(psTest));

Sample Introspection Test

Detect Rhino Engine Version

The following example demonstrates how to detect the version of the Rhino JavaScript engine as an introspection test, that is executed inside the Orchestrator..

/**
 * Checked with Aria Automation 8.18.1
 */

const jsCodeDetectRhinoVersion = "\n\
outputs = {};\n\
const context = org.mozilla.javascript.Context.enter();\n\
try {\n\
  const rhinoVersion = context.getImplementationVersion();\n\
  System.log(rhinoVersion);\n\
  const languageVersion = context.getLanguageVersion();\n\
  System.log(\"Language version: \" + String(languageVersion));\n\
  outputs.status = \"done\";\n\
  outputs.error = null;\n\
  outputs.rhinoVersion = rhinoVersion;\n\
  outputs.languageVersion = languageVersion;\n\
} catch (exception) {\n\
  System.error(String(exception));\n\
  outputs.status = \"incomplete\";\n\
  outputs.error = String(exception);\n\
  outputs.rhinoVersion = null;\n\
  outputs.languageVersion = null;\n\
} finally {\n\
  org.mozilla.javascript.Context.exit();\n\
}\n\
return outputs;\n\
";

const rhinoVersion = System.getModule("de.stschnell").testAction(
    "de.stschnell",
    "detectRhinoVersion",
    null,
    jsCodeDetectRhinoVersion,
    null,
    null
);

System.log(JSON.stringify(rhinoVersion));

Here is the result, which is returned as a JSON object.
Hint: For reasons of clarity, some passages have been shortened and replaced with three dots [...].

{
  "status": "done",
  "error": null,
  "createAction": {
    "id": "953a9bb8-880f-461c-927c-40fb591ae815",
    "input-parameters": [],
    "output-type": "Properties",
    "name": "detectRhinoVersion",
    "module": "de.stschnell",
    "version": "0.1.0",
    "script": "\noutputs = {};\n ... \nreturn outputs;\n"
  },
  "executeAction": {
    "value": {
      "properties": {
        "property": [
          {
            "key": "rhinoVersion",
            "value": {
              "string": {
                "value": "Rhino 1.7 release 4 2012 06 18"
              }
            }
          },
          {
            "key": "languageVersion",
            "value": {
              "number": {
                "value": 0
              }
            }
          },
          {
            "key": "status",
            "value": {
              "string": {
                "value": "done"
              }
            }
          },
          {
            "key": "error"
          }
        ]
      }
    },
    "type": "Properties",
    "execution-id": "df222b4c-1c0e-421a-889f-6a28c8885ecd"
  },
  "logAction": {
    "logs": [
      {
        "entry": {
          "origin": "system",
          "short-description": "... Rhino 1.7 release 4 2012 06 18",
          "time-stamp": "2026-06-20T09:34:30.449+00:00",
          "time-stamp-val": 1781948070449,
          "severity": "info"
        }
      },
      {
        "entry": {
          "origin": "system",
          "short-description": "... Language version: 0",
          "time-stamp": "2026-06-20T09:34:30.452+00:00",
          "time-stamp-val": 1781948070452,
          "severity": "info"
        }
      }
    ]
  },
  "deleteAction": true
}


result of an action introspection test in vcf automation orchestrator

Conclusion

The approach presented here breaks with the traditional mindset in infrastructure testing. Instead of treating the Orchestrator purely as a passive platform, it elevates the action itself into an active quality assurance tool. By leaving no technical debt or artifacts on the test system, this method completely isolates the test logic from the executing infrastructure. It validates both the custom code and the underlying orchestration platform, providing an efficient way to execute a deterministic self-test. Consequently, this approach is one interesting perspective of platform validation.

References