How to process the data sent by satellite — working with binary data as a part of PW-Sat2Tomasz ŁuczakBlockedUnblockFollowFollowingJan 16Over a year ago we have started working on an application whose goal was to enable amateur radio enthusiasts all over the world to participate in the PW-Sat2 project.
The problem is that, as the satellite orbits around the globe, we can communicate only in a time window when it passes somewhere above our heads.
The idea was to gather the HAM radio community scattered all over the world so that they could help collecting data.
Together with the PW-Sat2 team we have implemented a solution:The PW-Sat2 team has created a ground station application — desktop app responsible for direct communication with the satellite and decoding data.
We have prepared a web application where all the data can be uploaded and then presented in a user friendly form.
The overall schema of communication was described in the previous post.
As the communication is very expensive we could not go with standard protocols — no REST, no HTTP, not even TCP/IP — we are operating at the level of pure AX.
25 frames.
AX.
25 frame formatThe AX.
25 frame format is well-known and very well-described format.
The specification can be found here.
The structure of an AX.
25 frame is as follows:Flag is a marker byte which tells us where the frame starts and where it endsAddress fields defines both Destination and Source of a frame.
In our case itControl fields defines what kind of frame is being passed.
As PW-Sat2 uses “Unnumbered frame” it gets value of 0x03PID is a protocol identifier, whether it is a TCP/IP frame or maybe something else.
In our case it has value 0xF0 which means “No layer 3 protocol implemented.
”Info field is the frame payload.
FCS is a frame-check sequence.
It should be calculated by both sender and receiver to ensure that the frame was not corrupted during the transmission.
Binary data encodingWhen working with binary data, before we even get to data type definition, we need to be aware of two things:Bit order in bytesByte order in the data streamFor bits ordering the general approach is to place bits from the most significant bit (MsBit or highest bit) to the least significant bit (LsBit or lowest bit) — this kind of ordering is also called big endian ordering.
In this schema we have:2 => 0000 00103 => 0000 00118 => 0000 1000In case of numbers that are at least two byte long there is no unique approach for bytes ordering — it is arbitrary.
Two most common approaches are big endian and little endian.
Big endian schema places the most significant byte first:256 => 00000001 00000000This schema is most often used for network communication.
On the other hand, little endian places the most significant byte last:256 => 00000000 00000001This form of notation is used by most modern processors.
Data typesVarying data length is a hassle.
In case we would like to transmit values 8 (1000) and 3 (11) we could encode it as 1000 11 but as a result the other side could read it as 35 (as the space is ignored).
The problem is how encode the “space” between 8 and 3 in a binary form.
The easiest way is to introduce data types with defined length.
In our case “three” is a short number so it can be encoded with one byte:3 => 0000 0011“Eight” is also a short, so we can encode it as:8 => 0000 1000Building a stream with values 8 and 3 gives us 0000 1000 0000 0011.
It took two bytes, but now the other side of the communication has a clear idea how to split the data stream into separate fields no matter if we send values: 3 which is 2 bits long, 8 which is 4 bits long or 255 which is 8 bits long.
As long as we are passing fields which length is some number of bytes it works fine, but what if we want to send a field that is shorter — boolean for instance.
In current schema we should send 0000 0001 for “true”.
The problem is that is not a very efficient approach.
Due to cost of communication the PW-Sat 2 team decided to use only number of bits which is absolutely required for certain data.
PW-Sat 2 telemetry formatThe satellite’s telemetry consists of 179 fields encoded with 1832 bits (229 bytes).
The communication protocol defines exactly how many bits each field requires.
The stream is build with little endian encoding with bits ordered from the most significant for each byte.
Let assume we want to transmit stream of:12 (8 bits) — 0000110013 (8 bits) — 000011015 (4 bits) — 010112 (4 bits) — 1100In our schema we would need to write three bytes:Bytes are written from left to right, but bits are filled out from right to left — that is why in the last byte 12 is on the left hand side of the last byte.
In this case we managed to fit all fields within the same byte.
But things can get more complicated.
Let’s consider the following stream (big endian):1609 (11 bits) — 110 0100100131 (5 bits) — 1111121 (5 bits) — 101011609 (11 bits) — 110 01001001This can be encoded using only 4 bytes:The first “1609” takes the whole first byte and three lowest bits from the second byte (the ones that are on the right hand side of the byte).
The second “1609“ takes tree highest bits of the 3rd byte (the ones that are on the left hand side of the byte) and a whole 4th byte.
SummaryProcessing this kind of binary data can be a daunting task.
In case of PW-Sat2 telemetry we are talking more about a bit stream than a byte stream.
You also need to keep in mind that individual bits, are not addressable directly.
In case when you are working with 179 fields it can be cumbersome to process such data.
The good news is that there is a tool called scodec which helped us a lot with this task.
Stay tuned for the next part, where we will show exactly how we parse PW-Sat2 telemetry.
Special thanks to Maciej Nowak, Maria Wąchal and Mikołaj Koziarkiewicz for help in creating this publication.
.