Cloud Trigger is running, but can't be detected by API call

Problem

When you configure an extremely strict firewall, Cloud Trigger might not be detected by an API call.

Solution

To resolve the issue:

  1. Block ports 9350 to 9354 and 5671 in the Windows local firewall. Ensure ports 443 and 80 remain open.

  2. Investigate the network rules of the local Automation server and all firewalls that may transit on the network. Sniff traffic on the Automation server to identify any blocked ports, IP ranges, or geo-blocking rules.

Cloud Trigger attempts to open communication with the Service Bus in the following order:

  1. Legacy service bus mode (ports 9350-9354)

  2. AMQP/with TLS (5671-5672 with port 80/443)

  3. HTTPS AMQP-over-WebSockets (port 443 only)

If ports 9350-9354 are open in the local Windows firewall, Cloud Trigger will try to connect to the Service Bus using this port range over TCP. Usually, you prefer HTTPS, and Cloud Trigger may appear to be running and able to connect to the Loftware Cloud, but the Cloud Trigger call returns a 404 error, indicating the device is not connected.

Steps to Resolve:

  1. Block ports 9350-9354 in the local Windows firewall.

  2. Block ports 5671-5672 in the local Windows firewall.

  3. Restart the Automation service and deploy the trigger. This will ensure it uses port 443.

The connection usually breaks with Service Bus in WebSocket mode:

  1. Automation runs Cloud Trigger, which opens a connection to Azure Service Bus (lmscloud-bus-trigger-production.servicebus.windows.net). This is normal and expected.

  2. The client sends a request to Cloud API (labelcloudapi.onnicelabel.com). This is normal and expected.

  3. Cloud API sends a request to the Cloud Trigger function, which attempts to establish a connection with the client running Cloud Trigger. This is normal and expected.

  4. The Cloud Trigger function can’t establish a connection because it doesn’t send connection data on the existing web socket connection. Instead, it opens a new connection with the Service Bus relay (e.g., g6-prod-db4-008-sb.servicebus.windows.net) to send data. This is not an expected behavior.

  5. Cloud Trigger function can’t send data to Cloud Trigger, resulting in a 404 error returned to Cloud API, indicating the device is not connected. This is not an expected behavior.

Allow connection to the Service Bus relays using either a large IP range based on public Service Tag data or wildcard hostnames. Example hostnames include:

  • g6-prod-db4-008-sb.servicebus.windows.net

  • g25-prod-db4-008-sb.servicebus.windows.net

  • g11-prod-db4-008-sb.servicebus.windows.net

Use the IPs within the range 52.169.0.0/16.

Wildcard Examples:

  • *-sb.servicebus.windows.net

  • *-prod-*-*-sb.servicebus.windows.net

Important

Microsoft provides the service bus relays, and Loftware has no control over them. The service bus relays are dynamic and constantly changing, so Loftware can’t provide a fixed IP or hostname. You must whitelist the subnet or wildcard entries as an outbound entry.

Note

You can sniff websocket traffic using tools like Fiddler to observe how the connection is established with relays to send payload data.