Proton BASIC Compiler - Proton USB Bootloader

  • Pic® Basic

  • Proton USB Bootloader

    USB Bootloader

    By JohnB and Tim
    The principal function this system fulfills is to bootload your code into a PIC® using the USB port. The bootloader uses the native USB functions of specific PIC® processors as opposed to other USB Bootloaders which use a USB to serial converter and appear on an attached PC as a com port.The code is written in Proton Basic and should be easy to understand and modify. The other half of the system is the PC end and for this JohnB has written an application that will take your hex code and send it, along with the appropriate commands, to the PIC® where it is loaded into the flash (program) memory.

    The basics of writing a boot loader in Proton Basic

    Before delving into the details of the bootloader code it is worth discussing the basic operations of writing code to flash memory and some of the design choices available. Writing code to the flash memory of a PIC® is done in 2 stages:
    • Clear a block of flash memory to enable it to accept new data
    • Write that data to PIC flash memory
    Clearing - PIC® flash memory is cleared in banks, generally 64 bytes in length; in Proton it can be done using the command CErase. [1]

    - Where address is the start of the bank you wish to erase.

    Thus the first step is to run through the memory erasing all the banks with CErase

    - To write data you use the command Cwrite. Once again data is written in blocks, this time 16 bytes.

    - where address is the start address in memory of the 1st data byte.

    NOTE you have to begin from the start of the 16 byte block on the boundary and you have to write the full number of bytes before the process will finish.

    Designing a bootloader

    There are a number of points to consider when designing a USB bootloader:
    • Where the bootloader should reside in memory
    • Interrupt handling
    • What USB protocol to use
    • How the bootloader will be accessed i.e. How to know when to run the bootloader
    • Resilience or what happens if the writing is interrupted?
    Location - There can only be 2 locations a bootloader can reside; the Start of Memory or the End. Most serial bootloaders reside at the top of memory. Typically they work as follows:

    At the PC, the loader modifies the incoming Hex and inserts a jump to the bootloader code at the top of memory. On start-up it can look for incoming data. If, after a set period, there is no data on the USART it returns control back to the main program. The main advantage of this configuration is that the user code requires no modification to run (apart from the couple of instructions inserted automatically by the PC app).

    This is fine for a Bootloader that uses the USART as the code required to read a USART is minimal. A USB driver, on the other hand, requires a significant amount of code and while it would be possible to place a copy of this at the top of memory, it is an unnesesary waste of code space. Thus USB bootloader code is best placed at the bottom of memory and the main code adjusted to suit. This can be achieved simply by inserting the following line in Proton.

    = Address

    Where address is the starting address of your code.

    - A further issue is interrupts, since a hardware interrupt always jumps to either $0008 or $0018 depending on the priority level. However this can be resolved with a few lines in the bootloader to redirect the calls to the right place in the main code.

    Choice of USB Protocol
    - There are 2 basic types of USB Proton protocol:

    The first is the HID which stands for Human Interface Device. This is a driverless system that requires no user intervention apart from waiting a few seconds while the PC does some initializations the first time you use it.

    The second is CDC which is Composite Serial Device. This will create a virtual serial port on the PC and any PC apps talk through it as if it was a serial port. There are a couple of disadvantages: a special driver is required which must be installed before using the connection. Also, you cannot be sure which serial port number will be assumed for the USB connection so it has to be manually selected by the user.

    For simplicity and all round ease of use the HID interface is the preferred protocol.

    Initiating the bootloader
    - Knowing when to run the Bootloader is a very fundamental issue. With a serial port you can stream data at the port even if it’s not connected. In the PIC, during the first 500mS or so following startup a serial Bootloader can decide if its needed simply by monitoring the serial port. USB on the other hand can take a long time to connect, especially the first time it’s used on a PC, making this approach impractical.

    The solution is ultimately down to the designer of the device and how it fits their application. There is more on this later when we talk about the hardware, but as an early tease, you can tell if the PC is connected to a PC using just 1 resistor.

    - When you load your new program onto the PIC® it can take a few seconds, especially if it’s a big program. Plenty of time for something to go wrong, like pulling the connection plug. There has to be a method where you can monitor that the code was written successfully.

    The solution is pretty simple. Set an EEPROM flag when you clear the memory and only clear it when it the program load was deemed successful by the PC loader application. On start up check this flag, if it is set then something went wrong and the probability is that application code is incomplete.

    The Proton Code

    The full code can be downloaded below but for now we will pick out some of the main points. You might find it useful to have the bootloader code open in PDS while you read the following basics:
    • This code is written for the PIC18F14K50. You can change it yourself and this is covered later. The PIC18F14K50 was chosen as it is a great little device and very easy to use.
    • It runs on an external 12Mhz Xtal and is PLL to 48Mhz then divided internally to run at 24mhz. 12Mhz is needed for the USB side to work internally and the rest is sorted using the fuse settings.
    • The detection of the USB connection is achieved by monitoring one of two lines on the USB interface. This is discussed more fully in the Hardware section.
    USB Descriptor - Every USB device has to have a descriptor file which, among many other things, carries the VID (Vendor ID) and PID (Product ID) number. In the bootloader this file is an include file and is declared as:


    This file uses the standard MicroChip USB bootloader VID and PID which you can use for personal use. However, if you plan to sell a product then you need to sort out your own VID and PID.The section to modify in the descriptor is this

    dt 0xD8, 0x04
    ; idVendor low byte, high byte
    dt 0x3C, 0x00 ; idProduct low byte, high byte

    Note how the Hex bytes are reversed so the VID is $04D8 and the PID is $003C

    There are other sections you may wish to modify as well, like the Serial Number. Look through the code to see what needs to be changed.

    Alternative PIC
    ®s - These are the main defines you will need to alter to make the code work with another PIC®.
    MainCodeStart = $0B80
    ; Main code start address / Keep on a 64 byte boundary

    This address is where your code will start, it must be a multiple of the FlashEraseSeg value. It will be used to create the declaration PROTON_START_ADDRESS = Address
    EndOfCodeSpace = $4000
    ; End of code space

    This is the address of the end of the memory in the PIC® so it knows where to stop erasing.

    FlashEraseSeg = 64
    ; The size of memory bank erased in one go
    Read the PIC® data sheet for this information or look in the PDS PPI file.

    FlashWriteBlock = 16
    ; The memory size Flash write needs to do in one block

    Again read the manual or look at the PPI file.

    = %00000011
    ; Set enable interrupts on port A to get the read on porta.0/1 to work

    The Data sheet on the PIC18F14K50 is missing some sections notably how to detect the input on the D+ line. To do this you have to enable interrupts on the port. In my set up if the line is high then you are not connected, when the USB line is connected the line is pulled low. It works 99.9% of the time but it’s always better to have a backup, like a button.

    USB Bootloader Commands

    There are 4 fundamental commands required to communicate with the bootloader. Each command comprises a 64 byte packet sent from the PC via USB. Its format comprises a 1 Byte command followed optionally by Address and Data as required.

    - This will erase all the flashram between MainCodeStart and EndOfCodeSpace. It also sets a flag at the top end of EEPROM to say flash was erased.

    - Data is sent with the following packet info Address as a DWord followed by 16 bytes of data.

    The data is written to memory and then read back to compare it against what was sent. If the data is OK it responds with the default acknowledge of $55. if there is a discrepancy then the Error message of $AA is sent.

    - All this really does is clear the EEPROM set when you did an erase.

    - Jumps to the beginning of the user code.

    NOTE - Every command sends back an acknowledgement of $55 to say message was actioned. (the exception being a failure to write to flash properly were it will send back $AA)

    Programming - The Bootloader can also write to EEPROM if there is any EData in the hex file. EEPROM data is recognised by its address and then uses the Ewrite command to load the EEPROM data.

    Modifying the code for your own use

    There are many areas that can be modified; here is a list some of them and how to do it.

    Changing the VID, PID Serial no etc. -
    It is pretty easy to change the VID and PID in the descriptor; however the PC loader application only works with the Microchip™ VID and PID so for practical purposes it can only be used for development or personal use.

    If you require a loader with your own VID and PID which can generate a standalone .EXE that can be passed on to customers to update their firmware then please contact us. We have products which offer this and a lot more such as customization and encryption etc.

    Adding your own commands
    - The USB interface does not have to be restricted to bootloading new firmware. It could be used to act as a conduit to send data back to a PC or to allow the PIC® application to be controlled by the PC. This method is used in my product the TL1000 It’s a simple matter to add commands to the command Select...Case list which will call routines in your application code. However, you will have to write your own PC application to handle these additional commands.

    Calling a Bootloader
    Function - This may sound odd but can be useful. E.g. There may be a routine in the Bootloader section you want access to from the main code to save you replicating that code in your main application.

    Place a GoTo in the Bootloader code at a fixed place that jumps to the routine you require. Then from your main code make a Call to that address.E.g. in your bootloader code

    (MainCodeStart - $12) ; space for redirection gotos GoTo YourCodeSub

    In your main code

    YourCodeSub = MainStartAddress - $12
    @ Call YourCodeSub
    ; Call your code in the Bootloader section

    It should be noted that if the routine is using variables they will affect the main routine's variables as well. In this case you can allocate the variables in a location not used normally by either set of code. For example:

    Commonvar1 as Word At __USBIn_Buffer ; Allocate space for common variables
    Dim Commonvar1 as Byte At Commonvar1 + 2

    If the same set of variables are used in both routines you can call a routine and access the data produced on its return. Lastly any modifications will require more room for the code, adjust it by modifying the declare

    MainCodeStart = $0B80
    ; Main code start address / Keep on a 64 byte boundary

    Note - Start high then work back once you are happy with the routine. Remember to alter the value in your main code.

    Writing Your Application Code

    In general you write your application code in exactly the same was as you would for any other PIC® application with a couple of minor provisos. Since the bootloader is located at the start of the code space your main code will need to sit above it.There is only one line of code needed to achieve this:-

    = Address

    Where address is the starting address of your code and the same value as “MainCodeStart” in the bootloaderThe only other thing you need to consider is that interrupts will take one instruction longer to reach your interrupt handler so finely tuned timer interrupts will need adjusting slightly.

    And really that is all!


    As mentioned above there will be instances where you need to know if the device is connected to a PC. This turns out to be pretty simple as there a several signals you can pick up on. There are 2 methods available depending on how the device will be powered:

    Device powered independently
    • Detect the USB connection via the USB power line
    • When using a separate power supply to power your device connect the USB power pin to a spare pin on your device.
    • On power up check for a high on this pin to determine the presence of the USB connection.

    Device powered from the USB connector
    • Detect a USB connection using D+
    • If you are using the USB connector as the power source for your device the above option is unavailable to you.
    • However, when connected to a USB port on a PC the D+ line is pulled low. By putting a weak pullup on the D+ line and connecting it to a pin you can test the line on start up. High not connected. Low connected.

    NOTE: This system works 99% of the time but it’s always better that there is a backup like a button.

    The USB Basic Pic® Bootloader

    Obviously the other half of the equation is the PC end application and we have that covered. Note - This is a development tool and not intended for commercial use. If you require an application capable of producing a standalone .exe complete with encryption and version control (as used to update the firmware on the TL1000 then contact either Johnb or Tim Box for more info.

    The PC Boot loader should be familiar to anyone who has used the Loader supplied with PDS. The controls are similar but without the Read and Identify functions of the PDS loader.
    The USB Basic PIC® Bootloader installs in PDS as an IDE plug-in. You first need to program the USB bootloader code into the device using a standard programmer. Thereafter just connect the board to the PC via the USB connection and the “USB Basic PIC® Bootloader” will detect the connection. When you run the loader from PDS it will automatically open the Hex file for the current page. You can choose an alternative hex file from File, Open menu.
    You can view the Code and EEPROM data for your application by clicking on the appropriate tabs. To program your device, click on the Program Icon or choose Program from the Action menu. You can choose what actions are taken when you program the device from the Options menu.

    USB Basic Pic® Bootloader Command list
    File Actions Options Help
    Open Ctl+ O Program Ctrl+ P Program Code About F1
    Recently used Erase Ctrl+ E Program Data
    Exit Ctl+X Run Ctrl+ R Run User Code after Programming

    [1] The Proton manual says nothing about the CErase or CWrite commands but after a little testing and looking at the code it is pretty simple to use.

    The full Bootloader Code in Proton

    ;=== Compiler related defines ==================================
        Device 18F14K50
        Create_Coff = On
        Xtal = 24
        Dead_Code_Remove = On  
        Optimiser_Level = 3 
        Unsigned_Dwords = 1
        ' descriptor file, 
        USB_Descriptor ""     
    ;=== interrupt based defines ===============================
    ;=== Chip Fuse settings ============================================
       CPUDIV = CLKDIV2 ; CPU System Clock divided by 2
       USBDIV = OFF ; USB Clock comes directly from the OSC1/OSC2 oscillator block; no divide
       FOSC = HS ; HS oscillator
       PLLEN = On ; Oscillator multiplied by 4
       PCLKEN = OFF ; Primary clock disabled <<<<<<<<<<<<
       FCMEN = OFF ; Fail-Safe Clock Monitor disabled
       IESO = On ; Oscillator Switchover mode enabled     <<<<<<<<<<<<<<<<<<<<<
       PWRTEN = On ; PWRTEN enabled
       BOREN = On ; Brown-out Reset enabled and controlled by software (SBOREN is enabled)
       BORV = 27 ; VBOR set to 2.7 V nominal
       WDTEN = OFF ; WDT is controlled by SWDTEN bit of the WDTCON register
       MCLRE = OFF ; RE3 input pin enabled; MCLR disabled
       HFOFST = On ; HF-INTOSC starts clocking the CPU without waiting for the oscillator to stablize.
       STVREN = On ; Stack full/underflow will cause Reset
       LVP = OFF ; Low Voltage Programming disabled
       XINST = OFF ; Enhanced CPU disabled
       CP0 = OFF ; Block 0 not code-protected
       CP1 = OFF ; Block 1 not code-protected
       CPB = OFF ; Boot block not code-protected
       CPD = OFF ; Data EEPROM not code-protected
       WRT0 = OFF ; Block 0 not write-protected
       WRT1 = OFF ; Block 1 not write-protected
       WRTB = OFF ; Boot block not write-protected
       WRTC = OFF ; Configuration registers (300000-3000FFh) not write-protected
       WRTD = OFF ; Data EEPROM not write-protected
       EBTR0 = OFF ; Block 0 not protected from table reads executed in other blocks
       EBTR1 = OFF ; Block 1 not protected from table reads executed in other blocks
       EBTRB = OFF ; Boot block not protected from table reads executed in other blocks
    ;=== SFR Alias's ===================================================
        Symbol NOT_RABPU = INTCON2.7    ' PORTA And PORTB Pull-up Enable Bit
        Symbol RABIF =  INTCON.0    ' RA And RB Port Change Interrupt Flag Bit(1)
    ;* Description  Edata defines                                  *
        Symbol FlashClearedStatus   = 255                                 ; Address in Eeeprom of the current status of the eeprom if cleared etc          
    ;* Description  General Hardware I/0 defines                   *
        Symbol USBDPlus = PORTA.0
        Dim FSR_0 As FSR0L.Word
        ' General Variables
        Dim Index1 As Byte
        Dim Index2 As Byte
        Dim PromAddress As Word
        Dim MemoryPointer As Word
        Dim TempB1 As Byte
        Dim TempB2 As Byte
    ; Firmware related vars
        Dim BootOk As Bit
        Dim DelayUSBValue As Byte
        Dim DelayCount As Byte 
    ; Main Pic retated defines
        Symbol MainCodeStart            = $0B80                         ; Main code start address / Keep on a 64 byte boundry
        Symbol EndOfCodeSpace           = $4000                         ; End of code space
        Symbol FlashEraseSeg            = 64                            ; The size of memory bank erased in one go
        Symbol FlashWriteBlock          = 16                            ; The memory size Flash write needs to do in one block
    ;   Command List
    ;   Below is the list of commands available and info on them
        Symbol EraseProgram             = $01                           ; Erase memory
        ; Bytes expected 1 (command) 
        ; Returns $55 via USB when done
        Symbol WriteFlash               = $02                       
        Symbol FinishWrite             = $03                            ; Finish the write
        ; Bytes expected 1 (command) 
        ; Returns $55 via USB then and writes to eeprom to finish the write sequence 
        Symbol RunMainCode             = $04                            ; Run the main code above the bootloader
        ; Bytes expected 1 (command) 
        ' USB buffer...
        Symbol USBBufferSizeTX  = 64
        Symbol USBBufferSizeRX  = 64
        Dim USBInBuffer[USBBufferSizeRX] As Byte                        ; this is the receiving buffer (pic side) aliased to the USB RAM space    
        Dim USBOutBuffer[USBBufferSizeTX] As Byte
        Dim USBCommandIn As Byte At USBInBuffer#0
        Dim OffsetReadAlias As Byte At USBinBuffer#1
        Dim EepromAddressAlias As USBInBuffer#1
        Dim StartAddressAlias As Dword At USBInBuffer#1  
        Dim ByteCountAlias As Byte At USBInBuffer#5 
        Dim DataStartAlias As Byte At USBInBuffer#6
        ' some useful flags...
        Dim PP0 As Byte System        ' USBPOLL status return
        Symbol CARRY_FLAG = STATUS.0  ' high if microcontroller does not have control over the buffer
        Symbol ATTACHED_STATE = 6     ' is USB attached
        Symbol TRNIF = UIR.3          ' low if USB Busy
         GoTo ProtonCodeStart
    ' ************************************************************
    ' * program starts here...                                   *
    ' ************************************************************
    ; Here we set up the pic as required and decide if were running USB Download mode or main code 
        OSCCON2.2 = 1                                                   ; Enable the Osc unit
        OSCCON = %01010000                                              ; ensure its on external osc
        All_Digital = 1
        ANSELH = 0                                                      ; Set up some analogue port settings
        ANSEL =  %00001000
        Clear                                                           ; Clear memory
        IOCA = %00000011                                                ; Set enable interrupts on port A to get the read on porta.0/1 to work
        NOT_RABPU = 0                                                   ; Enable pullups for port a and b
        WPUA = %00001000                                                ; Enable it for RA3 the button input
        WPUB = 0                                                        ; No pullups on port b 
        DelayMS 100                                                     ; A delay for all to settle
    ' ************************************************************
    ' * wait for USB interface to attach                         *
    ' ************************************************************
       GoSub IsBootOk                                                   ; Check if were ok to run now
       DelayMS 100                                         
       If USBDPlus = 0 Then GoTo USB_Enumerate
       If BootOk = 0 Then GoTo USB_Enumerate
       GoTo RunMain
        IOCA = %00000000                                                ; Turn of the interrupts on PortA.0/1 thus disabling the read on the pins
        While 1 = 1
            Repeat                                                        ' \
                USBPoll                                                    '   Wait for the USB interface to become attached
            Until PP0 = %00000110                                        ' /
            GoTo USBMode
        GoTo USBMode
        @ GoTo MainCodeStart                                            ; All is ok so run main code 
    ' ************************************************************
    ' * receive data from the USB bus                            *
    ' ************************************************************
            USBIn 1, USBInBuffer, Auto
        Until STATUS.0 = 0
    ' ************************************************************
    ' * transmit Data                                            *
    ' ************************************************************
        If TRNIF = 1 Then GoTo USBLoop
            USBOut 1, USBOutBuffer, USBBufferSizeTX                                
        Until STATUS.0 = 0     
    ' ************************************************************
    '  Write the Data From the usbbuffer into flash ram
    '  $20, address 4 bytes, payload 
    ' ************************************************************   
        PromAddress = StartAddressAlias
        Index2 = ByteCountAlias - 1   
        If StartAddressAlias.Byte2 = $F0 Then GoTo WriteEepromValue     ; We do a jump to this routine and the code will return to the sender
        If PromAddress < MainCodeStart Then                             ; If the address were being requested to write to is not in the correct place then we just return                                                                                                                                       
            GoTo ResultOkExit
        FSR_0 = VarPtr DataStartAlias                                    ; point to the incoming data
        GoSub WriteToProm                                               ; Write that into the memory block and into Prom
    ' ************************************************************
    '  Now Read the Data Back to confirm it was saved Ok
    ' ************************************************************ 
        PromAddress = StartAddressAlias
        Index2 = ByteCountAlias
        FSR_0 = VarPtr DataStartAlias                                    ; point to the incoming data
        Index1 = 0
        Repeat                                                          ; Loop and test what's in memory with what is supposed to be there                                      
            TempB1 = CRead PromAddress
            TempB2 = POSTINC0
            If TempB1 <> TempB2 Then                                    ; If the data is not what was expected                                               
                GoTo SendBadResultandReturn                                ; Then send a fail result
            Inc PromAddress
            Inc Index1
        Until Index1 >= Index2
    ; If we get here data was written Ok so say so and return 
        GoSub USBOK_Responce
        GoSub USBNotOK_Responce
    ; end of WriteFlashData sub
    ' ************************************************************
    '  Write over that Memory block the required data 
    '  Then write that to Prom
    ' ************************************************************
        MemoryPointer = PromAddress                                     ; Point to 16 byte boundary in memory were we want to load our data
        Index1 = 0
            TempB1 = POSTINC0
            CWrite MemoryPointer,[TempB1]  
            Inc Index1
            Inc MemoryPointer   
        Until Index1 >= FlashWriteBlock                                 ; keep writting until the full 16 byte block of ram is written to Prom
    '' ************************************************************
    ''  System check to see if the last flash write went well
    '' ************************************************************ 
        If ERead FlashClearedStatus = "W" Then                        ; Read in from eeprom the current write status 
            BootOk = 1        
            BootOk = 0                                  
    ' ************************************************************
    '  Erase all of program memory above the boot loader line
    ' ************************************************************    
    ; First we write to eeprom we have started the write/clear phase
        EWrite FlashClearedStatus, ["C"]
        ; Now clear all of Flash above the bootloader
        PromAddress = MainCodeStart
            CErase PromAddress
            PromAddress = PromAddress + FlashEraseSeg
        Until PromAddress >= EndOfCodeSpace 
        Clear USBOutBuffer
        GoSub USBOK_Responce
    ' ************************************************************
    '  Write to Eeprom the required values
    ' ************************************************************  
        Index2 = USBInBuffer#5 - 1
        FSR_0 = VarPtr DataStartAlias
        TempB1 = EepromAddressAlias
        Index1 = 0
            EWrite TempB1,[POSTINC0]
            DelayUSBValue = 5
            GoSub DelayUSB
            Inc Index1 
            Inc TempB1
        Until Index1 = Index2
        GoSub USBOK_Responce
    ' ************************************************************
    '  A simple safe method of delaying and keeping the usb connection live
    '  ' It will be longer than the required delay
    ' ************************************************************
        DelayCount = 0
            DelayMS 1 
            Inc DelayCount
        Until DelayCount >= DelayUSBValue
    ; Send OK back on USB
        USBOutBuffer#0 = $55
        GoSub DoUSBOut
    ; Send back Not ok responce
        USBOutBuffer#0 = $AA
        GoSub DoUSBOut
    ;=== This is the main USB mode loop =========================
        While 1 = 1                                                     ; Main Loop Runs here
            GoSub DoUSBIn
            Clear USBOutBuffer      
            Select USBCommandIn
                Case EraseProgram                                       ; Erase Program memory
                    GoSub EraseProgramData                                              
                    GoSub USBOK_Responce 
                Case WriteFlash                                         ;  Write Y bytes from Prom at X,X,                       
                    GoSub WriteFlashData
                Case FinishWrite                                        ; Set the appropriate flags to say it was a successful firmware update           
                    EWrite FlashClearedStatus, ["W"]       
                    GoSub USBOK_Responce 
                Case RunMainCode                
                    GoSub USBOK_Responce
                    @ Call MainCodeStart                                ; Run the main code above the Bootloader
    Org  MainCodeStart + $8
    Org MainCodeStart + $18

    A Blinking LED example

        Device 18F14K50    
        Create_Coff = On
        Xtal = 24
        Symbol MainCodeStart            = $0B80                         ; Main code start address / Keep on a 64 byte boundry
        PROTON_START_ADDRESS = MainCodeStart                            ; Tell Proton were to compile to
        Dim LED As PORTB.4                                              ; The led
        All_Digital = 1
        While 1 = 1
            High LED
            DelayMS 1000
            Low LED
            DelayMS 1000
    The code, this article etc can be downloaded from HERE