Connecting...

W1siziisimnvbxbpbgvkx3rozw1lx2fzc2v0cy9zawduawz5lxrly2hub2xvz3kvanbnl2jhbm5lci1kzwzhdwx0lmpwzyjdxq

How to process the data sent by satellite — working with binary data as a part of PW-Sat2

W1siziisijiwmtkvmdevmjqvmtuvndcvmdgvntg5l3blegvscy1wag90by0yntyxntiuanblzyjdlfsiccisinrodw1iiiwiotawedkwmfx1mdazzsjdxq

Previously SoftwareMill began working on an application with the goal to allow radio enthusiasts globally to take part in the PW-Sat2 project which is a student satellite project. 

The aim was to collect and process the data, they have done this by implementing a solution. Want to know what that solution was? Have a read of this article by Senior Software Engineer, Tomasz Łuczak!

 

'Over a year ago we have started working on an application whose goal was to enable amateur radio enthusiasts all over the world to participate in the PW-Sat2 project. The problem is that, as the satellite orbits around the globe, we can communicate only in a time window when it passes somewhere above our heads. The idea was to gather the HAM radio community scattered all over the world so that they could help to collect data. Together with the PW-Sat2 team we have implemented a solution:

  • The PW-Sat2 team has created a ground station application — desktop app responsible for direct communication with the satellite and decoding data.
  • We have prepared a web application where all the data can be uploaded and then presented in a user friendly form.
The overall schema of communication was described in the previous post. As the communication is very expensive we could not go with standard protocols — no REST, no HTTP, not even TCP/IP — we are operating at the level of pure AX.25 frames.
 

AX.25 frame format

The AX.25 frame format is well-known and very well-described format. The specification can be found here. The structure of an AX.25 frame is as follows:

  • Flag is a marker byte which tells us where the frame starts and where it ends
  • Address fields defines both Destination and Source of a frame. In our case it
  • Control fields defines what kind of frame is being passed. As PW-Sat2 uses “Unnumbered frame” it gets value of 0x03
  • PID is a protocol identifier, whether it is a TCP/IP frame or maybe something else. In our case it has value 0xF0 which means “No layer 3 protocol implemented.”
  • Info field is the frame payload.
  • FCS is a frame-check sequence. It should be calculated by both sender and receiver to ensure that the frame was not corrupted during the transmission.
 

Binary data encoding

When working with binary data, before we even get to data type definition, we need to be aware of two things:

  • Bit order in bytes
  • Byte order in the data stream
For bits ordering the general approach is to place bits from the most significant bit (MsBit or highest bit) to the least significant bit (LsBit or lowest bit) — this kind of ordering is also called big endian ordering. In this schema we have:
  • 2 => 0000 0010
  • 3 => 0000 0011
  • 8 => 0000 1000

In case of numbers that are at least two byte long there is no unique approach for bytes ordering — it is arbitrary. Two most common approaches are big endian and little endian.

Big endian schema places the most significant byte first:

  • 256 => 00000001 00000000
This schema is most often used for network communication.
On the other hand, little endian places the most significant byte last:
  • 256 => 00000000 00000001

This form of notation is used by most modern processors.

 

Data types

Varying data length is a hassle. In case we would like to transmit values 8 (1000) and 3 (11) we could encode it as 1000 11 but as a result the other side could read it as 35 (as the space is ignored). The problem is how encode the “space” between 8 and 3 in a binary form. The easiest way is to introduce data types with defined length. In our case “three” is a short number so it can be encoded with one byte:

  • 3 => 0000 0011

“Eight” is also a short, so we can encode it as:

  • 8 => 0000 1000

Building a stream with values 8 and 3 gives us 0000 1000 0000 0011. It took two bytes, but now the other side of the communication has a clear idea how to split the data stream into separate fields no matter if we send values: 3 which is 2 bits long, 8 which is 4 bits long or 255 which is 8 bits long.

As long as we are passing fields which length is some number of bytes it works fine, but what if we want to send a field that is shorter — boolean for instance. In current schema we should send 0000 0001 for “true”. The problem is that is not a very efficient approach. Due to cost of communication the PW-Sat 2 team decided to use only number of bits which is absolutely required for certain data.

 

PW-Sat 2 telemetry format

The satellite’s telemetry consists of 179 fields encoded with 1832 bits (229 bytes). The communication protocol defines exactly how many bits each field requires. The stream is build with little endian encoding with bits ordered from the most significant for each byte. Let assume we want to transmit stream of:

  • 12 (8 bits) — 00001100
  • 13 (8 bits) — 00001101
  • 5 (4 bits) — 0101
  • 12 (4 bits) — 1100

In our schema we would need to write three bytes:

Bytes are written from left to right, but bits are filled out from right to left — that is why in the last byte 12 is on the left hand side of the last byte. In this case we managed to fit all fields within the same byte. But things can get more complicated. Let’s consider the following stream (big endian):

  • 1609 (11 bits) — 110 01001001
  • 31 (5 bits) — 11111
  • 21 (5 bits) — 10101
  • 1609 (11 bits) — 110 01001001

This can be encoded using only 4 bytes:

The first “1609” takes the whole first byte and three lowest bits from the second byte (the ones that are on the right hand side of the byte). The second “1609“ takes tree highest bits of the 3rd byte (the ones that are on the left hand side of the byte) and a whole 4th byte.

Summary

Processing this kind of binary data can be a daunting task. In case of PW-Sat2 telemetry we are talking more about a bit stream than a byte stream. You also need to keep in mind that individual bits, are not addressable directly. In case when you are working with 179 fields it can be cumbersome to process such data.

The good news is that there is a tool called scodec which helped us a lot with this task. Stay tuned for the next part, where we will show exactly how we parse PW-Sat2 telemetry.'

 

This article was written by Tomasz Łuczak and posted originally on SoftwareMill Blog.