Understanding USP Records

The TR-369 User Services Platform (USP) is a powerful standard for managing and controlling connected devices. A key component of USP is the USP Record, which plays a crucial role in the communication between devices.

In this post, we will explore what USP Records are and how they function.

A USP Record is defined as the Message Transfer Protocol (MTP) payload. It is like a USP Message in an envelope(it is a USP Record in this case), with the sender-receiver information.  It encapsulates a sequence of datagrams that make up the USP Message. In addition to encapsulating the USP Message, the USP Record also provides additional metadata needed for integrity protection, payload protection, and delivery of fragmented USP Messages.

Let's see the USP Record proto-buff definition from the official usp proto-buffer file:

message Record {
  string version = 1;
  string to_id = 2;
  string from_id = 3;
  PayloadSecurity payload_security = 4;
  bytes mac_signature = 5;  //MAC or Signature
  bytes sender_cert = 6;

  oneof record_type {
    NoSessionContextRecord no_session_context = 7;
    SessionContextRecord session_context = 8;
    WebSocketConnectRecord websocket_connect = 9;
    MQTTConnectRecord mqtt_connect = 10;
    STOMPConnectRecord stomp_connect = 11;
    DisconnectRecord disconnect = 12;
  }

  enum PayloadSecurity {
    PLAINTEXT = 0;
    TLS12 = 1;
  }
}

Fields within a USP Record

A USP Record contains several fields, each serving a specific purpose. Here are some of the fields contained within a USP Record:

1. Version: This required field represents the version (Major. Minor) of the USP Protocol (i.e., “1.0” or “1.1”).
2. To_id: This required field represents the receiving/target USP Endpoint Identifier.
3. From_id: This required field represents the originating/source USP Endpoint Identifier.
4. Record_type: This required field contains one of the types given below.

Security-Related Fields:

this indicates the protocol or mechanism used to secure the payload (if any) of the USP Message.

1. PayloadSecurity: This optional field is an enumeration of type PayloadSecurity. When the payload is present, this indicates the protocol or mechanism used to secure the payload (if any) of the USP Message.
2. Mac_signature: This optional field is used when integrity protection of non-payload fields is performed. It is the message authentication code or signature used to ensure the integrity of the non-payload fields of the USP Record.
3. Sender_cert: This optional field is the PEM encoded certificate, or certificate chain, of the sending USP Endpoint, used to provide the signature in the mac_signature field, when integrity protection is used and the payload security mechanism doesn’t provide the mechanism to generate the mac_signature.

NoSessionContextRecords and SessionContextRecords in USP

Two key components of USP are the NoSessionContextRecord and SessionContextRecord.

NoSessionContextRecord

A NoSessionContextRecord is a type of USP Record that encapsulates a USP Message but does not include any session context. This means that the message is not part of a larger conversation or session between two USP Endpoints. The only field included in a NoSessionContextRecord is the payload, which contains the USP Message itself.

Let's see the NoSessionContextRecord proto-buff definition from the official usp proto-buffer file:

message NoSessionContextRecord {
  bytes payload = 2;  ///YOUR PAYLOAD IS IN HERE as bytes, it is the USP Message
}

SessionContextRecord

Unlike a NoSessionContextRecord, a SessionContextRecord is part of a larger conversation or session between two USP Endpoints. This type of record includes several fields:

Let's see the SessionContextRecord proto-buff definition from the official usp proto-buffer file:

message SessionContextRecord {
  uint64 session_id = 1;
  uint64 sequence_id = 2;
  uint64 expected_id = 3;
  uint64 retransmit_id = 4;
  PayloadSARState payload_sar_state = 5;
  PayloadSARState payloadrec_sar_state = 6;
  repeated bytes payload = 7;    ///Here is your payload

  enum PayloadSARState {
    NONE = 0; 	    //No segmentation
    BEGIN = 1;      //Begin segmentation
    INPROCESS = 2;  //Segmentation in process
    COMPLETE = 3;   //Segmentation is complete
  }
}

1. Session_id: This field is the Session Context identifier. It's required and unique to each session.
2. Sequence_id: This field is the Datagram sequence identifier. It's used only for the exchange of USP Records with an End-to-End (E2E) Session Context. The field is initialized to 1 when starting a new Session Context and incremented after each sent USP Record.
3. Expected_id: This field contains the next sequence_id the sender is expecting to receive, which implicitly acknowledges to the recipient all transmitted datagrams less than expected_id. It's used only for the exchange of USP Records with an E2E Session Context.
4. Retransmit_id: This optional field is used to request a USP Record retransmission by a USP Endpoint to request a missing USP Record using the missing USP Record’s anticipated sequence_id. It's used only for the exchange of USP Records with an E2E Session Context. Will be received as 0 when no retransmission is requested.
5. PayloadSARState: This optional field is an enumeration of type PayloadSARState. When the payload is present, it indicates the segmentation and reassembly state represented by the USP Record.

When to Use SessionContextRecord or NoSessionContextRecord

The choice between using a NoSessionContextRecord or a SessionContextRecord depends on whether the USP Message is part of a larger conversation or session. If the message is standalone, a NoSessionContextRecord is used. If the message is part of a session, a SessionContextRecord is used. If you are unsure, it is most probable NoSessionContextRecord.

MQTTConnectRecord

The MQTTConnectRecord is used when an MQTT connection is established. This record type includes two fields:

Let's see the MQTTConnectRecord proto-buff definition from the official usp proto-buffer file:

message MQTTConnectRecord {
  MQTTVersion version = 1;
  string subscribed_topic = 2;

  enum MQTTVersion {
    V3_1_1 = 0;  // Represents MQTT v3.1.1, a.k.a. v4 in the MQTT Spec
    V5 = 1;
  }
}

1. Version: This required field represents the MQTT protocol version used by the USP Endpoint to send this Record. The valid values are V3_1_1 (0) and V5 (1).
2. Subscribed_topic: This required field represents an MQTT Topic where the USP Endpoint sending this Record can be reached (i.e., an MQTT Topic it is subscribed to).

STOMPConnectRecord

The STOMPConnectRecord is used when a STOMP connection is established. This record type also includes two fields:

Let's see the STOMPConnectRecord proto-buff definition from the official usp proto-buffer file:

message STOMPConnectRecord {
  STOMPVersion version = 1;
  string subscribed_destination = 2;

  enum STOMPVersion {
    V1_2 = 0;
  }
}

1. Version: This required field represents the STOMP protocol version used by the USP Endpoint to send this Record. The only valid value is V1_2 (0).
2. Subscribed_destination: This required field represents a STOMP destination where the USP Endpoint sending this Record can be reached (i.e., a STOMP destination it is subscribed to).

WebSocketConnectRecord

A WebSocketConnectRecord is a type of USP Record that is used when a WebSocket connection is established. This record type does not contain any fields. It serves as a signal that a WebSocket connection has been made.

Let's see the WebSocketConnectRecord proto-buff definition from the official usp proto-buffer file:

message WebSocketConnectRecord {
  // An empty message
}

DisconnectRecord

The DisconnectRecord is a type of USP Record that is used to signal the termination of a session.  It serves as a signal that a session has been terminated.

Let's see the DisconnectRecord proto-buff definition from the official usp proto-buffer file:

message DisconnectRecord {
  string reason = 1;
  fixed32 reason_code = 2;
}

This record type also includes two fields:

1. Reason: Optional. A code identifying the reason for the disconnect.
2. Reason Code: Optional. A string describing the reason for the disconnect.

References:

1. TR-369 USP Specification, Broadband Forum, https://www.broadband-forum.org/tr-369
2. USP Record Definitions, GitHub Repository, https://github.com/BroadbandForum/usp