Autoscale problem

oneto · Post by **oneto** » Sat Feb 21, 2004 2:10 pm

Hello,
Plotting data in double precision, it seems that sometimes the autoscale rounds the Y limits wrongly. I didn't succeeded to find a rule, but I suspect that could be linked to the axis values still in single precision ?
By the way, thanks for the double precision, it really helps me !
Jean-Louis Oneto

Post by **DPlotAdmin** » Sat Feb 21, 2004 4:18 pm

Jean-Louis,

Plotting data in double precision, it seems that sometimes the autoscale rounds the Y limits wrongly.

That's possible. An example would be helpful, just send me a file when you have the time.

If you allow DPlot to set the extents and the tick mark intervals, what it should be doing is finding a tick mark interval that's some power of 10 times 1,1.5,2,2.5,3,4,5,6, or 8 that gives roughly 10 intervals and includes the entire extents of the data.

I didn't succeeded to find a rule, but I suspect that could be linked to the axis values still in single precision

The FFT routines still use single precision, so DPlot first converts input to single precision, then converts the FFT output to double precision. Same thing applies to plugins and data passed to DPlot via DDE. All of these will be converted to double precision eventually, but I doubt this is the source of what you see as a problem.

By the way, thanks for the double precision, it really helps me !

You're quite welcome. It should have been done this way from the start, but back when men were men and dinosaurs roamed the earth I was more concerned about memory usage.

Post by **DPlotAdmin** » Sat Feb 21, 2004 6:33 pm

Jean-Louis,
Thanks for the example file. You are correct, there's a sloppy roundoff problem in there that's now fixed. Using the default scaling your minimum value of 0.269414956190 results in the lower limit being 0.2694149 rather than 0.269415.