Proton BASIC Compiler - A Simple Asynchronous Packet Protocol

  • Pic® Basic

  • A Simple Asynchronous Packet Protocol

    This article develops a method of achieving an asynchronous packet protocol with low overheads and easy to understand code.

    Projects that use communications are sometimes better using asynchronous data transfer. Systems like Ethernet require special hardware, cabling and chips. Asynchronous data can be used fairly simply to make your own network. Most PIC® chips have one or two USARTs in them. A rigid packet structure makes data integrity easier to control. The author has successfully implemented a radio packet network using these techniques.

    Most computer networks use packets of data. These packets usually conform to a rigid structure, the data within the packets can be received and actioned because the order of byte/bits is predefined and agreed upon by every station. For example, each packet starts with the destination address, usually followed by the sender's address. In this way several nodes (stations) can communicate. Let’s invent a protocol.

    Our packet will be a number of bytes, each packet will start with the byte address of the intended recipient.

    Something like this: DESTINATION, SENDER, data (1 byte)

    The reason the DESTINATION is first is so that each station can discard this packet straight away if it is not the intended recipient, thus avoiding wasted time decoding the rest of the packet. The SENDER is the address (byte) of the sending station.

    So with a three byte packet, we can send and receive a single byte of data to a remote station, maybe a command or report to start a motor for example. So far so good.

    If your network is made from long lengths of wire, or a radio link, there is a good chance that your data will be messed up by noise at some point. If your stations are allowed to transmit whenever they feel the urge, there is also a chance that two will transmit at the same time and both their data packets will be corrupted. Your motor will start when you least expect it etc. So how can we fix this? We are using USARTs and these devices all have the option to add parity checking. Would this help us? Slightly, but what if two bits got inverted by noise? The parity would still be “correct” would it not, but the data would be corrupt? We need a more robust method of detecting errors.

    We can use a checksum. This effectively adds up every byte of the packet before it is transmitted, the receiver then performs the same addition. If the result is as expected, the transmission is probably OK. (There is always a very small chance that an error will get through but that is too complicated to cover here. Assume the method is foolproof.) We will stick this checksum at the end of the packet. You will see how this is done later, it’s pretty simple using Proton.

    We now have a four byte packet: DESTINATION, SENDER, data, checksum (each field is one byte)

    Now think about some software to receive a packet. We will use interrupts and the packets will come in as a stream of four bytes. It is usual to send the whole packet at once rather than one byte now and then. If our network is busy it could happen that we get out of step on the receive side and treat (say) the second byte of the packet as the first. It would be nice to have a marker that means “this is the start of a packet”.

    If you look at the ASCII table (your Proton compiler has one on the toolbar) you will see that the first 32 values have names like Start of Text, End of Text and names that suggest they belong to some sort of data handling scheme. You would be right to think that – they are intended to help you build transmission systems, just like we are doing here! We are going to wrap our packets with two of these characters: Start of Text and End of Text. They have the values 2 and 3. We will define them in our Proton code as:


    The actual values do not matter, as long as we are consistent in our use of them. (These transmission control characters were chosen by the designers of ASCII so they don’t conflict with ASCII digits and alphabetic characters.) USARTs don’t try to interpret these characters in case you were worrying about that – they just transmit and receive groups of bits.

    Our packet now looks like this: STX, DESTINATION, SENDER, data, ETX, checksum

    It is now six bytes long but fairly bullet-proof as far as transmission errors go. The rule is “If the checksum fails on reception, throw the packet away, don’t assume anything in it is correct”, remember that. Notice also that the checksum byte comes last, after the ETX. It is normal to have it as the very last thing so it protects the whole packet including the ETX from errors during transmission.

    At this stage it might help to see the forming of the checksum and the transmission of a packet in Proton (all variables are BYTEs):

    chksum=STX ^ destination ^ sender ^ data ^ ETX
    HRSOut STX, destination, sender, data, ETX, chksum

    Have a good look at this and make sure you understand what is happening. “^” means exclusive OR in Proton (often written as XOR). If you are not sure what it does, spend some time and find out, it is very useful for this sort of thing. You are forming the exclusive OR of all the bytes in the packet, then bolting the result on the end of the packet. The receiver does the same thing and the result, if all is well, should be zero. In other words, if you exclusive OR all six bytes together, the answer should be zero. Prove this to yourself with a pencil and paper. It is assumed that you know what HRSOUT does and that you know how to set baud rate etc!

    Our packet is thus launched onto whatever medium you are using. I did this for a radio network using XRF modules (more on those later). We now need to think about receiving the packet in one of our nodes (stations). You could arrange for the packet to arrive in a buffer then set about deciphering what has been received. But there is a neater and better way (in my opinion), using the SELECT CASE structure of Proton. Remember that we are receiving a packet that must conform to a strict format (that we designed earlier). If anything arrives that we were not expecting, scrap the packet and wait for the next one. By using interrupts we can devise a crafty way of receiving a packet and verifying that it is valid all under the cloak of interrupts. The main program doesn’t have to do anything to receive a packet and it will be notified that a valid packet has arrived and can then process it. If the packet is damaged or invalid, it is quietly thrown away in the interrupt routine. If you are a bit frightened by interrupts, follow this through and try it for yourself, it’s not so difficult.

    Let’s write down what we need the code to do for each character as it arrives.

    Each character comes into the USART and triggers an interrupt. Add this to our running total of exclusive OR for the checksum. The number on the left below is the character number that has just arrived (1 is the first etc):

    1) Is this an STX character? If not then throw the packet out and start again, all packets MUST start STX. Otherwise move the expected character number on one. (In other words we are now expecting the second character of the packet, number 2 in this list.)

    2) Is this packet intended for this station DESTINATION? If not, throw it away as above. Otherwise step the next expected char number on one (to number 3).

    3) Remember where it came from (SENDER). Step the next expected char on one (to number 4).

    4) Store the “data” character somewhere to process later. Step the next expected etc..

    5) Is this byte an ETX? If not, scrap the packet. Otherwise step the next……

    6) Is the checksum now zero, if not, scrap the packet. Reset the pointer to the start point for the next packet. Mark that we have a good packet ready for action.

    Remember that all the above happens under interrupts, one character at a time. What follows is a simple State Machine. Now let’s code that in Proton. We have initialised pkt_so_far to zero during our main start-up. In this example there is only one interrupt source - the USART receiver:

     Context Save                 ;this is our interrupt routine
         RC2IF=0                  ;clear the interrupt flag             
         rx_chr=RCREG2            ;remember char from UART
         chksum=chksum^rx_chr     ;update checksum
       Select pkt_so_far          ;runs from zero. Increments 1 each byte
         Case 0                   ;first byte must be STX
          If rx_chr=STX Then      ;if it is start a new packet
           pkt_so_far=1           ;so we do Case 1 next
           chksum=STX             ;initialise the checksum and move on
         Case 1                   ;must be our address, if not, start again
          If rx_chr=Address Then  ;if us then move on
           Inc in_pkts            ;one more in packet (but may be bad etc)
           pkt_so_far=0           ;if not for us then start again
         Case 2
          Sender=rx_chr           ;the senders address
          pkt_so_far=3            ;move on 1 to Case 3 next
         Case 3                   ;data
          Function=rx_chr         ;store the “data” byte
          pkt_so_far=4            ;move on to Case 4 next
         Case 4
          If rx_chr=ETX Then      ;ETX must be followed by the checksum
           pkt_so_far=5           ;go to Case 5 next
           pkt_so_far=0           ;if not ETX then start again
           Inc bad_pkt            ;count bad packets for diagnostics   
         Case 5
           pkt_so_far=0           ;ready for next time regardless
           If chksum=0 Then       ;the checksum should now be zero
            got_pkt=1             ;it’s a good packet, tell main code      
            Inc bad_pkt           ;count bad packets, and ignore it
     Context Restore              ;restore regs and carry on 
    (Old versions of the compiler may need you to save some system variables that SELECT CASE uses.)

    Make sure you fully understand what is happening in the code above before we move on. In particular note that SELECT CASE only does one of the CASE statements per pass.

    The main code examines the got_pkt flag and if it is set, it means there is one packet ready to work on. It also means that if another packet should come along then it will overwrite the current values. You could add a test such that if you found a new STX, and got_pkt was 1 (from the previous packet), then ignore this new packet. That sort of thing is up to you.

    Now let’s look at what can go wrong in more detail. You have a MASTER control station and a SLAVE station. The MASTER can either ask the SLAVE to do something (turn on a lamp say) or it can request the current status of things that the SLAVE controls (which lamps are on). Say the MASTER wants to turn on LED number 2 on the SLAVE: the packet could look like this:

    STX, SLAVE, MASTER, 2, ETX, chksum

    This packet gets transmitted and noise corrupts the packet. The receiver (SLAVE) sees what looks like a good packet but the checksum fails, so it throws the packet away. We are now in a situation where the MASTER has commanded the SLAVE to do something, but it never will because it has thrown the packet away (the SLAVE has). Think about how this problem can be solved.

    The SLAVE could return a packet to the MASTER like this:

    STX, MASTER, SLAVE, 2, ETX, chksum

    This acknowledges the receipt of the command, and tells the MASTER which lamps are on. This would seem to be a good solution don’t you think? Well how about this scenario: MASTER tells SLAVE to turn on LED2 as before. SLAVE gets the packet without error and sends back its packet acknowledging the command. But noise corrupts the packet from SLAVE to MASTER. So the MASTER sits there forever waiting for a packet that will never arrive. We have improved things but we need to go a bit further.

    One way round this is to use time-outs. The MASTER sends a command and sets a timer going. If it gets the expected reply then all is well and we can carry on with the next step. But if the timer goes off, something has going wrong: either the SLAVE never got the packet, or the reply from the SLAVE got corrupted and lost.

    This is looking more promising. But consider this:
    • MASTER tells SLAVE to take x units of money from our bank account, also MASTER sets a timer going.
    • SLAVE gets the command and transfers the requested amount, it then acknowledges the command back to MASTER.
    • Acknowledgement gets corrupted and lost.
    • Timer goes off in MASTER so MASTER now knows something has gone wrong. MASTER repeats command to transfer same amount from our bank account.
    So we now get twice the expected amount transferred!!

    Welcome to the world of network protocols. It really is a world in its own right and I am not going to attempt to duplicate what has been done hundreds of times already. Google “network protocols” and be prepared to have some fun. I hope this has given you enough to let you carry on and do your own thing. If you want more bytes in your packet, do it. There are very few rules except common sense, you are in charge, you decide what happens.

    Just to give you a few ideas that might help:
    • Start with just two nodes, get them working first then add some more. You will have to work out how to interconnect several TX, RX signals from your stations.
    • Decide early on if you want a MASTER and SLAVES where the slaves never talk until requested by the master, or the system is all SLAVES where anybody can talk when they want to.
    I mentioned XRF radio modules earlier. These are complete radio links, you simply send/receive them RS232 data and they transmit it to another XRF unit some distance away (mine run at 868MHz). They are totally transparent to ASCII data and are a joy to use. You’ll need a 3.3V supply for them. Google XRF Radio Module.

    Good luck with your networking software. If you get something working that you think might benefit other members of this forum, please share it by writing an article for the Wiki or telling us in a post. As usual, if you spot any errors please let me know.