Contour Plott export and general data questions

[walleye3313]

David,

don't know if you recall, but i am working with GPS/Sonar data to make contour plot for lake bottoms. you helped me with my app last fall and now all is working well and i am ready to go to the next step in the project.

right now i can open the gps/sonar units output file convert to lat,lon,depth and make a plot in dplot jr. i want to add two features: join together two data sets from two different recording sessions and exporting the plot into a format that can be downloaded into the gps/sonar unit. the first is easy, the second i'm not sure of yet.

so here goes:

for the joining of data feature i can simply add the new data onto the end of the old data and plot the whole thing. simple and easy. but, if the data set gets too large then time complexity may bite me in the butt. how big of a file can i realistically plot? i have been testing with files of less than 50k and of course it plots faster than i can blink. but what if i start getting 256 Mb? or more, even 10 times more? 256 Mb in one session is the current working limit of the gps/sonar unit i am using.

now i am also thinking that there may be a lot of "repeated" data in subsequent sessions in the same area. so i am thinking about some kind of optimizing of the data set. i'll try to explain: the gps data has a resolution of DD MM.mmm so 1/1000 of a minute or about 1.5 meters. so if you think about a cartesian plane with a major division of .001 min then your location could be anywhere on that plane. if you happen to be on an intersection then your position is reported exacly, if you happen to be inside of a square then your position is reported to be the nearest corner to your true location. thus i could record the same coordinates for two actual position that are really nearly 3 meters apart. so i am trying to decide how to sort out that margin of error and what data to keep, toss, average or ??? what are your thoughts on this? if the time complexity is a non-issue then i could just leave it all in there and let the plotting function "average" it all into the contours. any input appreciated.

also recall that this is just for fishing, so a modest amount of "let it slide" is certainly OK.

on the exporting idea: i would like to be able to take the contoours and convert them back into a series of gps coordinates that could then be loaded into the gps unit for reference on the boat. can i get that type of data out of dplot in some way? i realize that i would have to limit the number of points used, but that is something that i can work out.

thanks for any help,

cheers
jerry

PS: if your interested i can send you the app to check out.

[DPlotAdmin]

Jerry,
For the past 6 months or so DPlot handles large collections of random 3D values much, much more efficiently. These changes were brought on after working with a user who was plotting 6 million data points. In general all data sets are now handled more efficiently. If your data has any sort of rectangular pattern to it, DPlot will generally figure that out and use that information to help in the triangulation, which speeds things up quite a bit more. Short version is - you shouldn't have any serious problems doing what you're talking about.

Repeated data: currently DPlot doesn't do any sort of averaging. It throws out all but the largest magnitude Z. I can make this an option if that is important to you.

Exporting: If I understand you correctly, you're wanting the X and Y coordinates of points that lie along a constant Z contour, right? If so there's no way to do that currently. This has been asked for in the past and will make it's way onto the to-do list, but I cannot promise a release date. If I've misunderstood what you're asking please give me more info.

And yes, I'm definitely interested in seeing what you're doing with your application. Thanks.

[walleye3313]

thanks for the reply david,

the file size will no longer be a worry to me, though 6 million data points are not out of reality for me either!



as to repeated data: that's interesting that it takes only the highest value. i'll have to do some more research with some friend at NRAO and see if they can start me in the right direction. i can work this issue out in an optimizing routine of some sort.


on the export question: you have it exactly. i still have to decode the data format used by the gps unit, unless the manufacturer will give it to me, before i need to go there. maybe i can find someone who is big on graphics to help me by working with the jpg of a plot and extracting the lines from the white.

i have your email addy so i will send you a beta version along with a raw data file to check out.

thanks
jerry

[walleye3313]

David,

i solved part of the problem of finding the coordinates of the black pixels, but that brings up a new question. since my data needs to be interpreted in sequence by the device, i.e. it is reading a trail of bread crumbs, i need to know wether there may be "gaps" in the contour lines when they are drawn by Dplot? i'll explain some more: if you use paint to draw some lines in a bitmap and then view the lines zoomed way in you see a series of pixels that are black on the white background and each pixel in a line is connected to the next either at the corner or along one side so they are "continuous" in a discrete sense (sorry math humor there).

is this what happens when dplot draws a contour line? will the corners or sides alway be "touching"? the reason i ask is that this is my strategy for locating the black pixels: look at each pixel starting at (1,1) and working across the y, then down the x until the first is found. once the first black is found i will then check the ajacent pixels in order clockwise starting from the pixel to the left (x+1,y) unitl the next pixel in the contour is located. if there are no gaps in the contours then this will allow me to follow the contour lines and record the relevant coordinates as i go. of course i will have to scale to bitmap to the lat/lon and then pick up where i left off to be sure i don't miss any contours but i have that worked out.

thanks
jerry

[DPlotAdmin]

Jerry,
There will never be any gaps in contour lines produced by DPlot. If you look at your plot with "Borders" under "Contour Options" checked the following might be a bit more clear. DPlot generates a triangular mesh from your input X,Y,Z points. Then for each triangle and a given contour level, it checks to see whether the contour value is within the Z extents of the triangle. If so it finds which 2 sides of the triangle are intersected and, using simple linear interpolation, the intersection point along each of those two sides. Then it just draws a line segment in the appropriate color between those two points. Simple stuff. A bit too simple to easily extract the information you want with the X,Y values across the entire surface for a given contour level.

If I haven't answered your question please let me know.

[walleye3313]

Thanks David,

I should be able to keep moving now.

i really appreciate your continued help on this project, though it is going slow being just a hobby i'm glad to be getting close.

cheers
jerry