Difference between revisions of "Technical topic: Hints to model complex packet encodings with ASN.1 and ACN"

From TASTE
Jump to: navigation, search
(Parameterized types, templates...)
(10 intermediate revisions by the same user not shown)
Line 1: Line 1:
In this page we are giving recipes and hints to help making use of ASN.1 and ACN to model complex packet structures. The example is based on the ECSS Standard named "PUS-C". This standard defines the packet structure for Telecommand and Telemetries used to exchange data between the Earth and Satellites.
+
= Introduction =
  
 +
In this page we are giving recipes and hints to help making use of ASN.1 and ACN to model complex packet structures.
  
== Context: Legacy Binary Encodings ==
+
* One example is based on the ECSS Standard named "PUS-C". This standard defines the packet structure for Telecommand and Telemetries used to exchange data between the Earth and Satellites.
 +
* Another example shows how to describe a Type-Length-Value kind of encoding, that is frequently found in communication protocols
 +
 
 +
 
 +
= Context: Legacy Binary Encodings =
  
 
ASN.1 is providing simple contructs to define data types which are independent from implementation languages. ASN.1 helps focusing on your ''application semantics'' rather than on how data will eventually be physically represented in memory, for example for network transmission.
 
ASN.1 is providing simple contructs to define data types which are independent from implementation languages. ASN.1 helps focusing on your ''application semantics'' rather than on how data will eventually be physically represented in memory, for example for network transmission.
Line 16: Line 21:
 
Well, this is not exactly true. ASN.1 has two options to deal with that family of problems (legacy encodings). The '''ECN''' language (''Encoding Control Notation'') is a full-blown ISO standard that allows to formally describe custom encoding rules. It is powerful, but also extremely complex. '''ACN''', on the other hand, is a light, free and open-source language that was developed by the European Space Agency to cover the same need but in a more ''user-friendly way'' (simple syntax and less features).
 
Well, this is not exactly true. ASN.1 has two options to deal with that family of problems (legacy encodings). The '''ECN''' language (''Encoding Control Notation'') is a full-blown ISO standard that allows to formally describe custom encoding rules. It is powerful, but also extremely complex. '''ACN''', on the other hand, is a light, free and open-source language that was developed by the European Space Agency to cover the same need but in a more ''user-friendly way'' (simple syntax and less features).
  
== The Packet Utilization Standard (PUS-C) ==
+
= Example 1: The Packet Utilization Standard (PUS-C) =
  
 
The PUS-C is a 600+ pages standard that was designed by European Space industries and agencies to have common grounds to control satellites in orbit. It details a number of services that have to be implemented in space systems, for example for monitoring the health of instruments in space, for patching memory areas, or for executing functions on-board.
 
The PUS-C is a 600+ pages standard that was designed by European Space industries and agencies to have common grounds to control satellites in orbit. It details a number of services that have to be implemented in space systems, for example for monitoring the health of instruments in space, for patching memory areas, or for executing functions on-board.
Line 30: Line 35:
 
Using ASN.1 you represent only the so-called ''application semantics''. This means you first remove all the fields that have no interest to the end user of the packet (padding fields, length fields, etc.).  This leaves you with a clean data structure that keeps only useful information that your application will process.
 
Using ASN.1 you represent only the so-called ''application semantics''. This means you first remove all the fields that have no interest to the end user of the packet (padding fields, length fields, etc.).  This leaves you with a clean data structure that keeps only useful information that your application will process.
  
(add ASN.1 example)
 
  
 
[[File:pus-cleaned.png]]
 
[[File:pus-cleaned.png]]
Line 65: Line 69:
  
  
   PUS-6 ::= CHOICE {
+
   PUS-6 ::= '''CHOICE''' {
 
       pus-6-1-memory-dump  Address,
 
       pus-6-1-memory-dump  Address,
 
       pus-6-2-memory-patch  Patch
 
       pus-6-2-memory-patch  Patch
 
   }
 
   }
  
   PUS-8 ::= CHOICE {
+
   PUS-8 ::= '''CHOICE''' {
 
       pus-8-1-execute-function Function-Identifier
 
       pus-8-1-execute-function Function-Identifier
 
   }
 
   }
  
   PUS-200 ::= CHOICE {
+
   PUS-200 ::= '''CHOICE''' {
 
       change-mode  Mode,
 
       change-mode  Mode,
 
       anything-else Whatever-Parameter
 
       anything-else Whatever-Parameter
Line 84: Line 88:
  
  
   PUS ::= CHOICE {
+
   PUS ::= '''CHOICE''' {
 
       manage-memory    PUS-6,
 
       manage-memory    PUS-6,
 
       execute-function PUS-8,
 
       execute-function PUS-8,
Line 94: Line 98:
 
Complete telecommand - with header, etc.
 
Complete telecommand - with header, etc.
  
   TC ::= SEQUENCE {
+
   TC ::= '''SEQUENCE''' {
 
       one-tc PUS
 
       one-tc PUS
 
   }
 
   }
 +
  
 
== Parameterized types, templates... ==
 
== Parameterized types, templates... ==
Line 112: Line 117:
  
 
[[File:pus-instance.png]]
 
[[File:pus-instance.png]]
 +
 +
 +
== Create the ACN structure ==
 +
 +
You see in a screenshot above a detailed ACN model of a PUS packets, with added bit patterns, etc. But to start with, a simple strucure is needed:
 +
 +
1) Add fields in the header part to contain the CHOICE discriminants:
 +
 +
  TC [] {
 +
      pus-type    INTEGER [size 8, encoding pos-int],    -- Added field
 +
      pus-subtype INTEGER [size 8, encoding pos-int],    -- Added field
 +
      one-tc      <pus-type, pus-subtype> []            -- Use the fields as parameters
 +
  }
 +
 +
2) Define the PUS encoding (top-level)
 +
 +
Associate the actual PUS service numbers from the standard to each CHOICE option:
 +
 +
  PUS <INTEGER: pus-type, INTEGER: pus-subtype> [] {
 +
      manage-memory    <pus-subtype> [present-when pus-type == 6],
 +
      execute-function <pus-subtype> [present-when pus-type == 8],
 +
      private-services <pus-subtype> [present-when pus-type == 200]
 +
  }
 +
 +
3) Associate the subtype
 +
 +
  PUS-6 <INTEGER: pus-subtype> [] {
 +
      pus-6-1-memory-dump        [present-when pus-subtype == 1],
 +
      pus-6-2-memory-patch      [present-when pus-subtype == 2]
 +
  }
 +
 +
  PUS-8 <INTEGER: pus-subtype> [] {
 +
      pus-8-1-execute-function  [present-when pus-subtype == 1]
 +
  }
 +
 +
  PUS-200 <INTEGER: pus-subtype> [] {
 +
      change-mode  [present-when pus-subtype == 1],
 +
      anything-else [present-when pus-subtype == 2]
 +
  }
 +
 +
Once this stucture is in place you can work on the individual encoding of the PUS parameters, on adding the padding bits, etc.
 +
 +
 +
== Call the compiler, generate ICD and check the result ==
 +
 +
Last but not least, you must verify your models using the compiler. You can also generate HTML tables and visualize the resulting encoding:
 +
 +
  $ asn1.exe -ACN -icdAcn out.html DataView.asn DataView.acn
 +
 +
Then open with a browser:
 +
 +
[[File:pus-icd.png]]
 +
 +
= Example 2: TLV Encoding =
 +
 +
== Introduction/BER Encodings ==
 +
 +
ASN.1's first and most known encoding format is named BER - Basic Encoding Rules. It is based on a TLV (Type-Length-Value) structure. Each basic ASN.1 type is assigned a tag (a number) and an encoding stream consists of a sequence of tags followed by the length of the value, and the value itself. This has the advantage of being easy to decode, and the receiver of a message does not need to embed the input grammar to be able to decode the fields.
 +
On the negative side, the encoding size is not optimal, and encoding/decoding speed is rather slow.
 +
 +
ASN1SCC does not support BER encodings. However it is possible using ACN to mimick such binary structures with a fine control over the encoded field sizes.
 +
 +
== Description of the problem ==
 +
 +
We want to describe a message that is either an error or a full message, including length and value fields.
 +
 +
In the binary stream the tag will be the discriminant:
 +
 +
  Tag    :  value 3=error, value 1=a valid message  use 2 bits
 +
  Length :  present only if tag value = 1,          use 14 bits
 +
  Value  :  present only if tag value = 1,          use the size specified in the Length field, but can be any type
 +
 +
== The ASN.1 representation ==
 +
 +
What is very important to understand about the ASN.1 and ACN approach: the data type (that will eventually be manipulated by the end user) '''shall only contain application semantics'''. This is fundamental: by no means the ASN.1 type shall mix up encoding information with the user data structure. The size of the fields is irrelevant at ASN.1 level - what matters is the range of values.
 +
 +
We will represent this message like this:
 +
 +
  My-Types DEFINITIONS ::=
 +
  BEGIN
 +
 
 +
    --  Define all possible valid messages
 +
    Valid-Message ::= CHOICE {
 +
          msg1 BOOLEAN,
 +
          msg2 INTEGER (0..65535)
 +
    }
 +
 
 +
    --  And then define the two options: either error, or a black box containing a valid message
 +
    TLV ::= SEQUENCE {
 +
        msg CHOICE {
 +
          error-msg NULL,
 +
          valid-msg SEQUENCE {
 +
              content OCTET STRING (CONTAINING Valid-Message)
 +
          }
 +
        }
 +
    }
 +
  END
 +
 +
== The ACN model ==
 +
 +
The description of the binary encoding for this structure resides in the ACN model - it does not mix up with the type definition.
 +
 +
In this model we will add the Tag and Length fields, which the user does not need to know about:
 +
 +
 +
:[[File:ClipCapIt-200910-140217.PNG]]
 +
 +
== Result ==
 +
 +
In order to check that the encoding is indeed compliant with the specification we can instruct ASN1SCC to generate documentation for it:
 +
 +
$ asn1.exe -icdAcn out.html -ACN tlv.asn tlv.acn
 +
 +
And open the resulting HTML file to see the result:
 +
 +
:[[File:ClipCapIt-200910-140443.PNG]]
 +
 +
 +
This shows that all the fields will have the required size and will be placed in the right order, as expected.
 +
ASN1SCC will generate a function to encode and decode this kind of messages in C or Ada/Spark with contracts.

Revision as of 10:11, 6 November 2020

Introduction

In this page we are giving recipes and hints to help making use of ASN.1 and ACN to model complex packet structures.

  • One example is based on the ECSS Standard named "PUS-C". This standard defines the packet structure for Telecommand and Telemetries used to exchange data between the Earth and Satellites.
  • Another example shows how to describe a Type-Length-Value kind of encoding, that is frequently found in communication protocols


Context: Legacy Binary Encodings

ASN.1 is providing simple contructs to define data types which are independent from implementation languages. ASN.1 helps focusing on your application semantics rather than on how data will eventually be physically represented in memory, for example for network transmission.

This is essential in any kind of distributed systems and embedded systems in general. When you want to send a meaningful set of data over a physical link, you can never count on how your local machine will store it. For example, if you use a type named "t_int16" in your code, you can never be sure that is you memory-dump it and send it as-is to a remote node that it will be properly decoded. Why? Because some machine would store the first byte first followed by the other one, while other would do the opposite. This is one example (endianness), but there are many others.

In some situations (actually, most of the time), you need to comply to some pre-established encoding rules, that are explicitly telling you, for a given protocol, how each packet shall be encoded.

ASN.1 is widely used and is popular because it provides built-in standard encoding rules that range from compact binary encoding (uPER, standing for unaligned Packed Encoding Rules) to ugly verbose XML encodings. Many telecommunication protocols are using these encoding rules. They allow users to rely on existing tools - you provide your ASN.1 grammar, and the tools provide encoders and decoders following the standard, therefore ensuring inter-operability.

However sometimes this is not sufficient... Some communication protocols (too many...) come with custom, proprietary encoding rules that are incompatible with any existing standard. In those situations, the usual approach consists in hand-writing binary encoding/decoding functions. ASN.1 may look far less relevant in that case, as automation can hardly be achieved...

Well, this is not exactly true. ASN.1 has two options to deal with that family of problems (legacy encodings). The ECN language (Encoding Control Notation) is a full-blown ISO standard that allows to formally describe custom encoding rules. It is powerful, but also extremely complex. ACN, on the other hand, is a light, free and open-source language that was developed by the European Space Agency to cover the same need but in a more user-friendly way (simple syntax and less features).

Example 1: The Packet Utilization Standard (PUS-C)

The PUS-C is a 600+ pages standard that was designed by European Space industries and agencies to have common grounds to control satellites in orbit. It details a number of services that have to be implemented in space systems, for example for monitoring the health of instruments in space, for patching memory areas, or for executing functions on-board.

A large part of the standard defines the bit layout of the telecommand and telemetry packets... Which makes it the perfect candidate for ASN.1 and ACN modelling. ACN was in fact originally created to support the automatic generation of PUS packets from C and Ada applications...

ASN.1 and ACN : the approach

The PUS packets look like this:

Pus-top.png

Using ASN.1 you represent only the so-called application semantics. This means you first remove all the fields that have no interest to the end user of the packet (padding fields, length fields, etc.). This leaves you with a clean data structure that keeps only useful information that your application will process.


Pus-cleaned.png

The missing fields are later added in the ACN model. ACN allows to link fields together, define their relative position in the packet, and specify how many bits are needed for each of them.

Pus-acn.png

The ASN.1 compiler then takes the ASN.1 and ACN models and generate code, documentation and test scripts. The code contains data structure (in C or in Ada) corresponding to the ASN.1 model, as well as functions to make a binary representation of the packets that is compliant to the description provided in the ACN model (representing in our case the complete PUS packets).

The following section gives an example to create proper ASN.1 and ACN models for PUS.

How to start?

There are of course several ways to deal with the PUS modelling. You have the possibility to define your own level of granularity in the models, and decide yourself what are the fields which matter to you for your application. However, a very important design driver should be to avoid repeating information and to keep the models as simple as possible - to ease the maintenance effort, namely.

The proposed methodology is the following:


1. PUS is based on high-level services that group functionalities of the same kind as "subservices" => mimic this structure using ASN.1 CHOICEs.

2. Create a aggregation of all high-level services

3. Create a SEQUENCE (record) container that contains additional fields possibly needed by PUS users

4. Some PUS services use project-specific data (constants, dedicated types) => use ASN.1 Templates (parameterized types) for that purpose

5. Create an ACN model to describe the actual PUS packets


Create the ASN.1 Structure

Create one CHOICE type for each PUS service


  PUS-6 ::= CHOICE {
     pus-6-1-memory-dump   Address,
     pus-6-2-memory-patch  Patch
  }
  PUS-8 ::= CHOICE {
     pus-8-1-execute-function Function-Identifier
  }
  PUS-200 ::= CHOICE {
     change-mode   Mode,
     anything-else Whatever-Parameter
  }

Create the aggregation

To choose between one of the PUS services, create a top-level CHOICE:


  PUS ::= CHOICE {
     manage-memory    PUS-6,
     execute-function PUS-8,
     private-services PUS-200
  }

Create the telecommand

Complete telecommand - with header, etc.

  TC ::= SEQUENCE {
     one-tc PUS
  }


Parameterized types, templates...

(This part is advanced, and can be skipped to start with...)

Some PUS services are very generic.. They have parameters that depend on the actual intance of the PUS for a given project. For example, the list of APID (node identifiers) or the maximum number of parameters for a specific service. ASN.1 is well equipped to support this kind of parameters at model level (using a template engine, similar in spirit to what you find with Ada's generics or C++ templates).

You have to define the placeholders:

Pus-parameterized.png

And later you'll need to provide the missing types/values according to your project need:


Pus-instance.png


Create the ACN structure

You see in a screenshot above a detailed ACN model of a PUS packets, with added bit patterns, etc. But to start with, a simple strucure is needed:

1) Add fields in the header part to contain the CHOICE discriminants:

  TC [] {
     pus-type    INTEGER [size 8, encoding pos-int],    -- Added field
     pus-subtype INTEGER [size 8, encoding pos-int],    -- Added field
     one-tc      <pus-type, pus-subtype> []             -- Use the fields as parameters
  } 

2) Define the PUS encoding (top-level)

Associate the actual PUS service numbers from the standard to each CHOICE option:

  PUS <INTEGER: pus-type, INTEGER: pus-subtype> [] {
     manage-memory    <pus-subtype> [present-when pus-type == 6],
     execute-function <pus-subtype> [present-when pus-type == 8],
     private-services <pus-subtype> [present-when pus-type == 200]
  }

3) Associate the subtype

  PUS-6 <INTEGER: pus-subtype> [] {
     pus-6-1-memory-dump        [present-when pus-subtype == 1],
     pus-6-2-memory-patch       [present-when pus-subtype == 2]
  }
  PUS-8 <INTEGER: pus-subtype> [] {
     pus-8-1-execute-function   [present-when pus-subtype == 1]
  }
  PUS-200 <INTEGER: pus-subtype> [] {
     change-mode   [present-when pus-subtype == 1],
     anything-else [present-when pus-subtype == 2]
  }

Once this stucture is in place you can work on the individual encoding of the PUS parameters, on adding the padding bits, etc.


Call the compiler, generate ICD and check the result

Last but not least, you must verify your models using the compiler. You can also generate HTML tables and visualize the resulting encoding:

  $ asn1.exe -ACN -icdAcn out.html DataView.asn DataView.acn

Then open with a browser:

Pus-icd.png

Example 2: TLV Encoding

Introduction/BER Encodings

ASN.1's first and most known encoding format is named BER - Basic Encoding Rules. It is based on a TLV (Type-Length-Value) structure. Each basic ASN.1 type is assigned a tag (a number) and an encoding stream consists of a sequence of tags followed by the length of the value, and the value itself. This has the advantage of being easy to decode, and the receiver of a message does not need to embed the input grammar to be able to decode the fields. On the negative side, the encoding size is not optimal, and encoding/decoding speed is rather slow.

ASN1SCC does not support BER encodings. However it is possible using ACN to mimick such binary structures with a fine control over the encoded field sizes.

Description of the problem

We want to describe a message that is either an error or a full message, including length and value fields.

In the binary stream the tag will be the discriminant:

  Tag    :  value 3=error, value 1=a valid message   use 2 bits
  Length :  present only if tag value = 1,           use 14 bits
  Value  :  present only if tag value = 1,           use the size specified in the Length field, but can be any type

The ASN.1 representation

What is very important to understand about the ASN.1 and ACN approach: the data type (that will eventually be manipulated by the end user) shall only contain application semantics. This is fundamental: by no means the ASN.1 type shall mix up encoding information with the user data structure. The size of the fields is irrelevant at ASN.1 level - what matters is the range of values.

We will represent this message like this:

 My-Types DEFINITIONS ::=
 BEGIN
 
    --  Define all possible valid messages
    Valid-Message ::= CHOICE {
          msg1 BOOLEAN,
          msg2 INTEGER (0..65535)
    }
 
    --  And then define the two options: either error, or a black box containing a valid message
    TLV ::= SEQUENCE {
       msg CHOICE {
          error-msg NULL,
          valid-msg SEQUENCE {
             content OCTET STRING (CONTAINING Valid-Message)
          }
       }
    }
 END

The ACN model

The description of the binary encoding for this structure resides in the ACN model - it does not mix up with the type definition.

In this model we will add the Tag and Length fields, which the user does not need to know about:


ClipCapIt-200910-140217.PNG

Result

In order to check that the encoding is indeed compliant with the specification we can instruct ASN1SCC to generate documentation for it:

$ asn1.exe -icdAcn out.html -ACN tlv.asn tlv.acn 

And open the resulting HTML file to see the result:

ClipCapIt-200910-140443.PNG


This shows that all the fields will have the required size and will be placed in the right order, as expected. ASN1SCC will generate a function to encode and decode this kind of messages in C or Ada/Spark with contracts.