Floating point strangeness


+ Reply to Thread
Results 1 to 7 of 7

Thread: Floating point strangeness121 days old

  1. #1
    Prolific Poster rcurl's Avatar
    Join Date
    Mar 2002
    Posts
    1,823
    Thumbs Up
    Received: 30
    Given: 53
    Total Downloaded
    2.16 GB

    0 Not allowed!

    Default Floating point strangeness

    I just recompiled code that was previously compiled about a year ago, and now I'm getting some very strange results from a simple formula that involves both words and floats. Here is the shortest snippet of code that will demonstrate the problem:
    Code:
     	Device = 18F46K80
    	Declare Xtal 16 	 'Using 16 Mhz internal clock 
    	Config_Start
       RETEN = OFF ; Ultra low-power regulator is Disabled (Controlled by REGSLP bit)
       INTOSCSEL = High ; LF-INTOSC in High-power mode during Sleep
       SOSCSEL = Dig ; Digital (SCLKI) mode
       XINST = OFF ; Disabled
       FOSC = INTIO2 ; Internal RC oscillator
       PLLCFG = OFF ; Disabled
       FCMEN = OFF ; Disabled
       IESO = OFF ; Disabled
       PWRTEN = On ; Enabled
       BOREN = On ; Controlled with SBOREN bit
       BORV = 3 ; 1.8V
       BORPWR = High ; BORMV set to high power level
       WDTEN = OFF ; WDT disabled in hardware; SWDTEN bit disabled
       MCLRE = OFF ; MCLR Disabled, RG5 Enabled
       STVREN = OFF ; Disabled
       Cp0 = On ; Enabled
       CP1 = On ; Enabled
       CP2 = On ; Enabled
       CP3 = On ; Enabled
       CPB = On ; Enabled
       CPD = OFF ; Disabled
       WRT0 = OFF ; Disabled
       WRT1 = OFF ; Disabled
       WRT2 = OFF ; Disabled
       WRT3 = OFF ; Disabled
       WRTC = OFF ; Disabled
       WRTB = OFF ; Disabled
       WRTD = OFF ; Disabled
       EBTR0 = OFF ; Disabled
       EBTR1 = OFF ; Disabled
       EBTR2 = OFF ; Disabled
       EBTR3 = OFF ; Disabled
       EBTRB = OFF ; Disabled
    Config_End
    
       Declare Bootloader off
       Declare Float_Display_Type = Fast
       
       OSCCON = %01110000  '16 mhz internal clock
       OSCCON2 =%00000000
       
            		
    ' ** Set direction of ports **
         ANCON0 = %00001111                 
    	 ANCON1 = %00000010                 
    	 SLRCON = %00000000                 
    	 CVRCON = %00000000                 
         TRISA = %10101111
    	 TRISB = %00010000
     	 TRISC = %00000000    
         TRISD = %00000000
         TRISE = %00001111
    
    ' ** Declare LCD settings **
            Declare LCD_DTPin PORTB.0
            Declare LCD_ENPin PORTD.7
            Declare LCD_RSPin PORTD.6
            
    ' ** Declare Variables **
        Dim PotVal As Word                 
        Dim Thresh As Float                
      				
    ' ** Declare constants **
         Symbol kvmult = 0.6616263
         Symbol minmult = 10740        
    
    Mainloop:
    
        PotVal = 32768  
        Thresh = kvmult * PotVal + minmult
        Print At 2,1,Dec1 Thresh, " ", Dec5 PotVal
        Print At 1,1,Dec5 kvmult, " ", Dec5 minmult
        DelayMS 500
        GoTo Mainloop
    What's happening is that as long as the word variable "potval" is below 32768 all is good, but when it hits 32768 the result, which should be 32420.5 becomes -10940.1. When potval is above 32768 the result is all over the place.

    I don't know the exact version of the compiler when this worked previously. Any thoughts as to what the problem might be?

    Thanks!

    -Rick
    Last edited by rcurl; 19th June 2019 at 22:28.

Attention

This valuable resource relies upon the very small amount of revenue generated by displaying online advertisements to our visitors.

The advertisements we display are relevant to this web site and your browsing history

Please consider supporting us by disabling your ad blocker.


Note: Some users have reported issues related to ad-blockers rendering parts of this wesite unusable,
where possible we will rectify the issues to enable you to use this resource with adblocking enabled.

If you can, please report issues in the forum area WebSite / Forum Issues


Thank you for your attention.

  • #2
    Senior Member tumbleweed's Avatar
    Join Date
    May 2011
    Posts
    473
    Thumbs Up
    Received: 58
    Given: 0
    Total Downloaded
    330.54 MB

    0 Not allowed!

    Default Re: Floating point strangeness

    It would appear that it's interpreting PotVal as a signed value even though it's declared as a 'word'.
    32768 would be 0x8000, so that's the smallest signed 16-bit value (-32767) if you're using 2's compliment notation.

    From this thread (post #12)

    "Therefore, you would not be able to load a floating point variable from a (16-bit) Word variable correctly if the Word variable held a value greater than 32767, otherwise, the Floating Point variable would actually hold a negative value."

    I'm not sure why something declared as an unsigned value would be used as a signed value.

  • Attention

    This valuable resource relies upon the very small amount of revenue generated by displaying online advertisements to our visitors.

    The advertisements we display are relevant to this web site and your browsing history

    Please consider supporting us by disabling your ad blocker.


    Note: Some users have reported issues related to ad-blockers rendering parts of this wesite unusable,
    where possible we will rectify the issues to enable you to use this resource with adblocking enabled.

    If you can, please report issues in the forum area WebSite / Forum Issues


    Thank you for your attention.

  • #3
    Administrator John Drew's Avatar
    Join Date
    Feb 2002
    Posts
    2,760
    Thumbs Up
    Received: 77
    Given: 33
    Total Downloaded
    4.30 GB

    0 Not allowed!

    Default Re: Floating point strangeness

    I agree with Tumbleweed.
    This needs an anomaly report.
    John

  • Attention

    This valuable resource relies upon the very small amount of revenue generated by displaying online advertisements to our visitors.

    The advertisements we display are relevant to this web site and your browsing history

    Please consider supporting us by disabling your ad blocker.


    Note: Some users have reported issues related to ad-blockers rendering parts of this wesite unusable,
    where possible we will rectify the issues to enable you to use this resource with adblocking enabled.

    If you can, please report issues in the forum area WebSite / Forum Issues


    Thank you for your attention.

  • #4
    Prolific Poster RGV250's Avatar
    Join Date
    Aug 2005
    Posts
    2,309
    Thumbs Up
    Received: 28
    Given: 7
    Total Downloaded
    801.38 MB

    0 Not allowed!

    Default Re: Floating point strangeness

    Oops, I only saw the first post

    Hi Rick,
    Was the earlier version before signed variables? as it looks like a 2s compliment issue.
    Try making PotVal a signed word or i would say a Dword.

    Regards,
    Bob
    Last edited by RGV250; 20th June 2019 at 11:50.

  • Attention

    This valuable resource relies upon the very small amount of revenue generated by displaying online advertisements to our visitors.

    The advertisements we display are relevant to this web site and your browsing history

    Please consider supporting us by disabling your ad blocker.


    Note: Some users have reported issues related to ad-blockers rendering parts of this wesite unusable,
    where possible we will rectify the issues to enable you to use this resource with adblocking enabled.

    If you can, please report issues in the forum area WebSite / Forum Issues


    Thank you for your attention.

  • #5
    Prolific Poster rcurl's Avatar
    Join Date
    Mar 2002
    Posts
    1,823
    Thumbs Up
    Received: 30
    Given: 53
    Total Downloaded
    2.16 GB

    0 Not allowed!

    Default Re: Floating point strangeness

    Quote Originally Posted by RGV250 View Post
    Was the earlier version before signed variables?
    Yes, it was.
    Quote Originally Posted by RGV250
    As it looks like a 2s compliment issue.
    Yep! I agree.
    Quote Originally Posted by RGV250
    Try making PotVal a signed word or i would say a Dword.
    Changing it to a Dword puts a band-aid on the problem, so it's at least working now.

    Thanks, Bob!

    -Rick

  • Attention

    This valuable resource relies upon the very small amount of revenue generated by displaying online advertisements to our visitors.

    The advertisements we display are relevant to this web site and your browsing history

    Please consider supporting us by disabling your ad blocker.


    Note: Some users have reported issues related to ad-blockers rendering parts of this wesite unusable,
    where possible we will rectify the issues to enable you to use this resource with adblocking enabled.

    If you can, please report issues in the forum area WebSite / Forum Issues


    Thank you for your attention.

  • #6
    Prolific Poster rcurl's Avatar
    Join Date
    Mar 2002
    Posts
    1,823
    Thumbs Up
    Received: 30
    Given: 53
    Total Downloaded
    2.16 GB

    0 Not allowed!

    Default Re: Floating point strangeness

    Quote Originally Posted by John Drew View Post
    I agree with Tumbleweed.
    This needs an anomaly report.
    Yes - I think so. I'm always hesitant to post an anomaly report because it's usually just me doing something stupid.

    Thanks, guys.

    -Rick

  • Attention

    This valuable resource relies upon the very small amount of revenue generated by displaying online advertisements to our visitors.

    The advertisements we display are relevant to this web site and your browsing history

    Please consider supporting us by disabling your ad blocker.


    Note: Some users have reported issues related to ad-blockers rendering parts of this wesite unusable,
    where possible we will rectify the issues to enable you to use this resource with adblocking enabled.

    If you can, please report issues in the forum area WebSite / Forum Issues


    Thank you for your attention.

  • #7
    Prolific Poster rcurl's Avatar
    Join Date
    Mar 2002
    Posts
    1,823
    Thumbs Up
    Received: 30
    Given: 53
    Total Downloaded
    2.16 GB

    0 Not allowed!

    Default Re: Floating point strangeness

    Les just uploaded 3.7.1.9 which fixed the problem. Thanks for the AMAZING support, Les!

    -Rick

  • Attention

    This valuable resource relies upon the very small amount of revenue generated by displaying online advertisements to our visitors.

    The advertisements we display are relevant to this web site and your browsing history

    Please consider supporting us by disabling your ad blocker.


    Note: Some users have reported issues related to ad-blockers rendering parts of this wesite unusable,
    where possible we will rectify the issues to enable you to use this resource with adblocking enabled.

    If you can, please report issues in the forum area WebSite / Forum Issues


    Thank you for your attention.

    Thread Information

    Users Browsing this Thread

    There are currently 1 users browsing this thread. (0 members and 1 guests)

       

    Similar Threads

    1. Floating point anomaly?
      By Mellbreak in forum Proton Plus Compiler v3
      Replies: 3
      Last Post: 3rd June 2016, 18:17
    2. Floating point anomaly?
      By joesaliba in forum Proton Plus Compiler v3
      Replies: 6
      Last Post: 20th January 2015, 22:27
    3. Floating point resolution
      By shantanu@india in forum Proton Plus Compiler v3
      Replies: 3
      Last Post: 9th January 2008, 13:41
    4. Floating point errors
      By Dave-S in forum Proton Plus Compiler v3
      Replies: 6
      Last Post: 4th June 2007, 21:20
    5. [SOLVED !] floating point precision
      By ryan.reeve in forum Proton Plus Compiler v3
      Replies: 14
      Last Post: 30th June 2006, 22:20

    Members who have read this thread : 39

    Actions :  (Set Date)

    You do not have permission to view the list of names.

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts