Generate->Y=f(X) X interval calculation off by 1

Found a bug? Tell us about it here

Moderator: DPlotAdmin

Post Reply
toddbatzler
Posts: 7
Joined: Fri Aug 28, 2009 8:58 am

Generate->Y=f(X) X interval calculation off by 1

Post by toddbatzler »

DPlot Version 2.2.6.5

Set from X = 0, to X = 0.049999, dx = 1e-006 and number of points reported is 50001. Generate waveform, edit data and waveform was generated up to and including X= 0.05, not 0.049999

Same behavior if numbers 0.1 to 0.199999 dx = 1e-6, but ok with 0.2 to 0.299999 or any subsequent interval where tenths digit is >1.
User avatar
DPlotAdmin
Posts: 2312
Joined: Tue Jun 24, 2003 9:34 pm
Location: Vicksburg, Mississippi
Contact:

Post by DPlotAdmin »

The trouble (in your example) is that even with double-precision floats, 49999 * 1.E-6 is equal to (roughly) 0.049999 - 6.9E-18. That's of course less than 0.049999, so a point is added. I'll modify the test to add a small (very small) bit of slop.
Visualize Your Data
support@dplot.com
Post Reply