Yet Another Cycling Forum

General Category => The Knowledge => GPS => Topic started by: Phil W on 09 June, 2016, 04:27:41 pm

Title: Slim down your bloated GPX track files
Post by: Phil W on 09 June, 2016, 04:27:41 pm
Are you feeling bloated, or at least are your GPX track files feeling bloated? Then read on.

Sian Lambert posted on the Audax FB group a few days back about her GPX file being rejected by the AUK mandatory route form because it was too large / bloated. It became clear not many know how to reduce the size of their GPX or even if they do they find it quite hard to get right.

I've taken something I've been doing for years to slim down GPX files and converted it into a simple one page web site / application.

All you need to do is select a GPX file on your computer, and a tolerance. A new slimmed down GPX based on the tolerance will be returned to you in zip format .

Some sample outputs for a 200km track

1m tolerance - .209KB (58KB zipped) about 3200 track points
 5m tolerance - 93KB (27KB zipped) about 1400 track points
 10m tolerance - 61KB (19KB zipped) and about 900 track points
 20m tolerance - 42KB (13KB zipped) and about 600 track points
 50m tolerance - 23KB (7KB zipped) and about 300 track points
 100m tolerance - 15KB (4KB zipped) and about 200 track points

I'm hosting it on a free server and it's free to use. Use at your own risk and I certainly recommend putting the slimmed down GPX back into a mapping program to make sure you are happy with the result.

Because it's a free server, performance will be what it is, but unless a great number of you hit it at the same time it should process your GPX in a timely manner.


https://simple-gpx.herokuapp.com
Title: Re: Slim down your bloated GPX track files
Post by: frankly frankie on 10 June, 2016, 09:13:06 am
Noting of course that '1m tolerance' exceeds the accuracy of the consumer GPS system that we use.

That looks very good and stripping out the timestamps and elevations really makes a big difference.  Of course that's not appropriate when dealing with a ridden tracklog - but for Planner-generated files this would work very well, if your main focus is reducing file size.   Mind, with GPX simply zipping usually achieves a huge reduction in file size - sometimes a tenfold reduction.
If your focus is a specific number of points (for example if you use an older Etrex, you need to target 500 points) - then I think Bikehike is easier to use.

I experimented:
A 200km Track made in Bikehike - contained 3070 trackpoints, 378Kb, 43.4Kb zipped.
Reduced to 1000 points in Bikehike - (actually 976 trackpoints) 120Kb, 15.5Kb zipped.
Reduced to 500 points in Bikehike - (actually 500 ) 62Kb, 8.5Kb zipped.  All these files include elevations and timestamps.

The first file (3070 points / 378Kb) run through your converter at its default setting - reduced to 887 points, 41Kb, 7.7Kb zipped.

Nice, clean minimal utility.  :thumbsup:
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 10 June, 2016, 03:31:33 pm
Yes 1m does exceed accuracy of GPS. I may well remove that option but it does still reduce bloat whilst essentially keeping the track mostly untouched unless it started with a ridiculous number of track points.

Yes this is for the tracks you plan to follow rather than recorded tracklogs where you wouldn't want to strip out timestamp and elevation data.

I'm toying with the idea adding the option to split long tracks into 50/100/200km sections with a suffix etc.  So you could feed it a 600km track, tick the split option and split distance, and it'd reduce your track points and GPX size and also split the track into 3 tracks say BCM-1 BCM-2 BCM-3 where the prefix BCM etc. is of your choosing.
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 10 June, 2016, 03:40:57 pm
Not just file size either, I sense my eTrex 20 is much quicker at processing the slimmed down tracks than the bloated versions that online maps produce. This is of course anecdotal :-)
Title: Re: Slim down your bloated GPX track files
Post by: Genosse Brymbo on 11 June, 2016, 02:49:46 pm
I'm not sure I understand your tolerance concept - surely a 200km track at 1m tolerance with about 3200 track points has a trackpoint at every 200*1000/3200 metres?  This is a trackpoint 62.5 metres, not every 1 metre.  So what does 1m tolerance mean if not the average distance between trackpoints?  Or have missed/miscalculated something?
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 11 June, 2016, 04:10:24 pm
I'm not sure I understand your tolerance concept - surely a 200km track at 1m tolerance with about 3200 track points has a trackpoint at every 200*1000/3200 metres?  This is a trackpoint 62.5 metres, not every 1 metre.  So what does 1m tolerance mean if not the average distance between trackpoints?  Or have missed/miscalculated something?

Yes you've misunderstood. It's not about the number of track points per metre.  It's about removing redundant track points from the track but still having a track that aligns to the original within the specified tolerance.

See https://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm and look at the animated diagram on the right hand side.  So lets for simplicity say I have a track with 3 track points.  I draw a line between the first and last track points. If the second track point is less than (or equal) x metres from that line then it is removed, if it further than x metres it is kept.  Thus keeping the aligment of the simplified track to within the tolerance.
Title: Re: Slim down your bloated GPX track files
Post by: rogerzilla on 12 June, 2016, 09:54:07 pm
GPS Babel does a pretty good job of this.  I used to use it to downsize GMaps Pedometer routes for the eTrex.  It is clever enough to only delete points where there are no bends or turns to be mapped, so on an archetypal stretch of Roman road it would just have a point at either end but it would closely map any wiggly bits on other roads.
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 13 June, 2016, 01:42:34 pm
GPS Babel does a pretty good job of this.  I used to use it to downsize GMaps Pedometer routes for the eTrex.  It is clever enough to only delete points where there are no bends or turns to be mapped, so on an archetypal stretch of Roman road it would just have a point at either end but it would closely map any wiggly bits on other roads.

It'll be using the same algorithm (or a close variant) my web app uses, but my app provides a much simpler and easier way of doing it, which is the point.

I've now added the option to optionally also split a long track down into either 50km, 100km, or 200km tracks.
Title: Re: Slim down your bloated GPX track files
Post by: aidan.f on 13 June, 2016, 07:42:07 pm
Works very  well, 600K route downloaded from RWGPS  1.8MB - put  thro' @5M accuracy comes  out  at an unzipped 132Kb  :) 
Title: Re: Slim down your bloated GPX track files
Post by: aidan.f on 13 June, 2016, 08:57:00 pm
Phil.  I  notice that you tag the file, quite right too! -
Quote
creator="Simple GPX https://simple-gpx.herokuapp.com ">
How about using this  to auto-compress zip and tag AUK copyright to GPX tracks when orgs. upload to the new AUKweb-site?
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 13 June, 2016, 10:55:17 pm
Phil.  I  notice that you tag the file, quite right too! -
Quote
creator="Simple GPX https://simple-gpx.herokuapp.com ">
How about using this  to auto-compress zip and tag AUK copyright to GPX tracks when orgs. upload to the new AUKweb-site?

I'm not involved in the new website and I'm sure they've tons of competing priorities to deal with already. No reason you couldn't submit your idea to the new IT manager though and see what he says. It would certainly simplify the workflow for many.
Title: Re: Slim down your bloated GPX track files
Post by: Oaky on 27 July, 2016, 12:07:01 pm
i think I found a bug (or perhaps there's something pathological in this particular RideWithGPS output).  I have a file that reduces to nothing when I upload it (default 10 metre setting).

input: 831 points, 36 miles (https://www.dropbox.com/s/7np2ap4ztory3i3/Fails.gpx?dl=1)
output: 2 pts, 0 miles (https://www.dropbox.com/s/q9sro0r5uyfodzk/Fails.zip?dl=1)
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 28 July, 2016, 11:16:22 am
i think I found a bug (or perhaps there's something pathological in this particular RideWithGPS output).  I have a file that reduces to nothing when I upload it (default 10 metre setting).

input: 831 points, 36 miles (https://www.dropbox.com/s/7np2ap4ztory3i3/Fails.gpx?dl=1)
output: 2 pts, 0 miles (https://www.dropbox.com/s/q9sro0r5uyfodzk/Fails.zip?dl=1)

Try again, should be fixed. 
Title: Re: Slim down your bloated GPX track files
Post by: Oaky on 28 July, 2016, 11:18:11 am
i think I found a bug (or perhaps there's something pathological in this particular RideWithGPS output).  I have a file that reduces to nothing when I upload it (default 10 metre setting).

input: 831 points, 36 miles (https://www.dropbox.com/s/7np2ap4ztory3i3/Fails.gpx?dl=1)
output: 2 pts, 0 miles (https://www.dropbox.com/s/q9sro0r5uyfodzk/Fails.zip?dl=1)

Try again, should be fixed.

Yep - working now!
Title: Re: Slim down your bloated GPX track files
Post by: Greenbank on 27 June, 2017, 08:02:10 pm
Phil.  I  notice that you tag the file, quite right too! -
Quote
creator="Simple GPX https://simple-gpx.herokuapp.com ">
How about using this  to auto-compress zip and tag AUK copyright to GPX tracks when orgs. upload to the new AUKweb-site?

AUK doesn't own the copyright though, it'll most likely belong to the organiser.
Title: Re: Slim down your bloated GPX track files
Post by: Oaky on 27 June, 2017, 09:54:11 pm
Phil's tool reduces the track complexity by removing unnecessary points (and stripping superfluous information e.g. elevation).  It's meant for simplification of tracks that are intended for navigational use in GPS units (some of which have  limits on the number of trackpoints for navigation).

zipping a file does a binary compression of the track, without changing the number of points.  you'd use this typically when transferring the track to soemone else, to save on bandwidth and/or storage space.  The track once unzipped would be identical to the original,  with the number of points unchanged etc.

(Incidentally,  Phil's tool does also zip the file, probably for exactly the reasons in the second paragraph above, but only after doing the clever stuff as per the first paragraph above).
Title: Re: Slim down your bloated GPX track files
Post by: Feanor on 27 June, 2017, 10:12:04 pm
Because the Garmin units can't use a ZIP file, they need the un-zipped GPX file.
The issue is not so much the size of the file in bytes; it's the number of trackpoints they contain.
That's what some of the older Garmins choke on.
Title: Re: Slim down your bloated GPX track files
Post by: frankly frankie on 28 June, 2017, 10:19:42 am
If you present a file already zipped does this cause issues? FWIW as a DIY org I can report that the auto zip function seems to have increased successful use of the form significantly.

Hi Andy, I'm glad to hear that the zipping appears to have improved the entry success rate.  That's my observation too, from looking at error logs generated whenever an entry fails.  Basically the entry form accepts files up to 5x larger than before and zips them, if they were not zipped before submission.  Files already zipped before submission are unaffected, that is they don't get double-zipped, and they are still subject to the stated size limit on the form.  Because zipping of GPX is highly efficient (much more than 5x), the best approach with a very large file is still to zip it before submission - which is one part of what Phil's very neat utility does.

I must say I have used Phil's utility a lot, in contexts nothing to do with Audax or DIYs - yes other tools such as GPS Babel can do similar things with more options, but I find this is an incredibly useful 'quick fix' to get to a clean and efficient GPX file. 
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 28 June, 2017, 10:44:12 pm
Because all edge units up to the 800 series and all etrex units up to 10/20/30 will truncate or crash if trying to navigate a track greater than 10,000 points. Ride with GPS and other online tools will typically spit out more than 10,000 points even for a 200km track. For even older units you can change the tolerance to drop below 500 points, or also decide to split a track into multiple tracks based on distance.

So it solves two problems,  10,000 track point  limit and size of files. With just zipping some ride with GPS GPX files are still in the Mb in size.  Plus it also gives the GPS track navigation and auk mandatory route comparison tool a lot less work to do. Efficiency and stability all round.
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 28 June, 2017, 10:54:28 pm
One other point.

I didn't build the tool for mandatory GPX track  submission as I built it long before such a thing existed. I just saw it could be used to solve that particular problem (too large a file submitted) for some, without it needing to be modified in any way.
Title: Re: Slim down your bloated GPX track files
Post by: jsabine on 28 June, 2017, 11:01:05 pm
It is true that Garmins can't use zip files and that older devices have a 500 trackpoint limit. Perhaps Phil's tool could be useful in this area but this isn't the subject of the OP. It is all about file size.

Eh? Phil's original post gives half a dozen examples of how choosing different accuracies affects the number of points, which is the only aspect of his tool I've been interested in.

As a case in point, the RWGPS tracks of days one and two of this weekend's Celtic Knot were about 13000 and 12000 points respectively, leading to them being truncated on my Etrex at about 290km each. Running them through the tool at default settings brought them each down to more like 2500 points - they opened faster on the device, and more importantly showed the track for the full length of the day.

It's definitely useful in this area, no perhaps about it.
Title: Re: Slim down your bloated GPX track files
Post by: Whitedown Man on 13 July, 2017, 10:19:24 am
Because all edge units up to the 800 series and all etrex units up to 10/20/30 will truncate or crash if trying to navigate a track greater than 10,000 points. Ride with GPS and other online tools will typically spit out more than 10,000 points even for a 200km track. For even older units you can change the tolerance to drop below 500 points, or also decide to split a track into multiple tracks based on distance.

So it solves two problems,  10,000 track point  limit and size of files. With just zipping some ride with GPS GPX files are still in the Mb in size.  Plus it also gives the GPS track navigation and auk mandatory route comparison tool a lot less work to do. Efficiency and stability all round.

Do TCX files suffer from the same problem, i.e. too many points for - in my case - an Edge Touring? I've just replaced my 810 (died of old age) with an Edge Touring. I never had an issue with anything up to and including 400km TCX files on the 810 but don't know whether the new device will cope.
Title: Re: Slim down your bloated GPX track files
Post by: αdαmsκι on 16 July, 2017, 05:54:33 pm
I used this yesterday to slim down the tracks for my ride today.  It was very easy to use and at no point was I unsure where I was going.

One feature that would be nice would be an option to maintain the elevation data. On my Etrex 20x I get high and low points, but with the elevation data removed from the track these didn't appear. It isn't too much a deal in Essex, but it's very useful when doing long climbs to know the top of the hill involves another X amount of vertical climbing. Ta.
Title: Re: Slim down your bloated GPX track files
Post by: Oaky on 16 July, 2017, 09:13:41 pm
I used this yesterday to slim down the tracks for my ride today.  It was very easy to use and at no point was I unsure where I was going.

One feature that would be nice would be an option to maintain the elevation data. On my Etrex 20x I get high and low points, but with the elevation data removed from the track these didn't appear. It isn't too much a deal in Essex, but it's very useful when doing long climbs to know the top of the hill involves another X amount of vertical climbing. Ta.

You can get (some) high/low points back if your map has elevation information.  These usually correspond to more significant high/lows compared with the ones from a track with elevation which often seemed to be every little lump.
Title: Re: Slim down your bloated GPX track files
Post by: αdαmsκι on 16 July, 2017, 09:23:28 pm
You can get (some) high/low points back if your map has elevation information.

I'm not sure if my map has this or not. I'll investigate.



These usually correspond to more significant high/lows compared with the ones from a track with elevation which often seemed to be every little lump.

Even if my map has elevation it won't have found anything significant today as I was in mid Essex.
Title: Re: Slim down your bloated GPX track files
Post by: Oaky on 16 July, 2017, 09:42:04 pm
These usually correspond to more significant high/lows compared with the ones from a track with elevation which often seemed to be every little lump.

Even if my map has elevation it won't have found anything significant today as I was in mid Essex.

I did say "more" significant. That was taking into account mid-Essex standards.  ;D ;D

I got fed up of the sheer number of high/lows tracks with elevation were generating here in the hallowed ACME heartlands, and so startd stripping the elevation.  The ones I get now, from the elevation data in my OSM maps, are far fewer, so I have more of my actual waypoints appearing in the "distance to next" field, in place of the minor lumps the Garmin was spamming me with before.
Title: Re: Slim down your bloated GPX track files
Post by: Manotea on 18 July, 2017, 07:31:35 pm
What would be useful is a utility that strips out all the extraneous data leaving just location, timing and elevation info in place, i.e., just those elements required for validation.
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 14 October, 2017, 02:27:53 pm
Simple GPX updated today.

https://simple-gpx.herokuapp.com/

It will round the lat / lon coordinates of the track down to 5 decimal places. I believe this is a precision of about 1 metre, rather than the misleading indicated precision and untold decimal places you find in source GPX tracks these days.

Options to retain the time stamp and / or elevation data in a track. Elevation data will be rounded to the nearest metre. The default is that this data will be stripped out.   If you intend to navigate a track then please don't choose to retain time stamp data. Some GPS units do not like it.
Title: Re: Slim down your bloated GPX track files
Post by: citoyen on 14 October, 2017, 02:56:14 pm
I've been using GPS Visualiser but will give this a go next time I need to reduce a GPX file. Sounds good.  :thumbsup:
Title: Re: Slim down your bloated GPX track files
Post by: frankly frankie on 14 October, 2017, 11:37:32 pm
That looks really, really, good.  All options very useful.  Thanks!  :thumbsup:
Title: Re: Slim down your bloated GPX track files
Post by: citoyen on 15 October, 2017, 09:20:56 pm
Just had a go, using a file exported from RideWithGPS. 525kb reduced to 36kb, using 10m tolerance and removing elevation data. Have checked the new file by uploading it back into RWGPS and it looks fine to me. Really simple and quick to use. Top work, Phil!  :thumbsup:
Title: Re: Slim down your bloated GPX track files
Post by: frankly frankie on 16 October, 2017, 09:01:14 am
It's excellent for de-bloating large track files (I've used it a lot over the last year or so, thanks Phil  :thumbsup: ), but not for reducing the point count to below a specified number - it doesn't offer that option.  For that specific purpose (which admittedly isn't needed much any more) I find BikeHike works better than most.
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 16 October, 2017, 03:17:44 pm
but not for reducing the point count to below a specified number - it doesn't offer that option. 

It doesn't but to get below the 500 track point limit you can use these rough guidelines for getting there.

100km route - set to 20m tolerance (suspect 15m tolerance would be near enough spot on but I don't offer that option)
200km route - set 50m tolerance
400km route - set 100m tolerance
Title: Re: Slim down your bloated GPX track files
Post by: Greenbank on 18 October, 2017, 08:29:20 am
A "below n trackpoints" option would still be nice (I know this wasn't the primary intention of the tool) but...

(The usual method is to use exactly the same algorithm it currently does but binary search based on the threshold until you find a threshold value that gives the required number of trackpoints in the output. More processing but it should be able to handle it).
Title: Re: Slim down your bloated GPX track files
Post by: frankly frankie on 18 October, 2017, 09:03:23 am
Most tools return a track with substantially fewer points than the target figure (for fairly obvious reasons) but BikeHike succeeds in meeting the target almost exactly.  Not sure if that's better or worse but I like it.
Title: Re: Slim down your bloated GPX track files
Post by: Somnolent on 18 October, 2017, 11:44:08 am
Just got around to trying Simple GPX on a recorded track (with timestamp & elevation retained) to check all is kosher before updating the advice in http://www.aukweb.net/resources/filesize/

Highly successful in reducing the number of trackpoints and file size ... but there is something not quite right with the file that means routevalidator not liking it.  Nor is Mapsource for that matter - another one that is known to be sensitive to exact compliance with the schema.

FF - number of failed uploads to PL01 this morning.  You may be able to see the problem?
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 18 October, 2017, 12:26:08 pm
Highly successful in reducing the number of trackpoints and file size ... but there is something not quite right with the file that means routevalidator not liking it.  Nor is Mapsource for that matter - another one that is known to be sensitive to exact compliance with the schema.

I had reversed the elevation and timestamp within each trackpoint which according to the GPX schema are in the sequence <ele>..</ele> then <time>...</time>.  Just corrected it and it now outputs them in the same sequence the GPX 1.1 schema states
Title: Re: Slim down your bloated GPX track files
Post by: Somnolent on 18 October, 2017, 12:57:38 pm
Highly successful in reducing the number of trackpoints and file size ... but there is something not quite right with the file that means routevalidator not liking it.  Nor is Mapsource for that matter - another one that is known to be sensitive to exact compliance with the schema.

I had reversed the elevation and timestamp within each trackpoint which according to the GPX schema are in the sequence <ele>..</ele> then <time>...</time>.  Just corrected it and it now outputs them in the same sequence the GPX 1.1 schema states

Works now  :thumbsup:
Title: Re: Slim down your bloated GPX track files
Post by: Greenbank on 18 October, 2017, 01:18:13 pm
The joy of XML is that you have to be very exact/correct in what you produce, but very lax in what you accept in.
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 18 October, 2017, 06:16:52 pm
I have now run my Wild Atlantic Way Audax tracklog through it at default settings keeping timestamo and elevation.  File comes out at 78kB zipped and 452kB un compressed.  That is a 2016km tracklog, which retained 9958 trackpoints.
Title: Re: Slim down your bloated GPX track files
Post by: frankly frankie on 18 October, 2017, 06:57:45 pm
Impressive.  5 points per kilometre I think is unnecessarily sparse though - maybe for a ultra ride like this it makes sense but in general I think around 20 points per km is a better target to aim at.

Otherwise its a far cry from the 'gold standard' of a raw recorded tracklog - even worse if you then start to add interpolation as suggested elsewhere.  It's bad enough that virtually all tracklogs are post-processed before submission anyway (up to Strava and down again, topping and tailing, joining two parts of a 600, etc etc).  Talk about fake data.  If I had written the checker software the very first thing it would red-flag would be a <creator> tag that indicated 'not a GPS'.

</rant>
Title: Re: Slim down your bloated GPX track files
Post by: Greenbank on 18 October, 2017, 07:43:46 pm
Until there are cryptographically signed trackpoints there's little point in worrying about whether data is real or fake.
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 28 October, 2017, 06:55:19 pm
The algorithm is now 3 dimensional in determining which points to keep.  So if you decide to retain elevation it will use that in the algorithm.

As a little extra I thought I'd get it to create a HMAC of its work

<gpx version="1.1" creator="Simple GPX https://simple-gpx.herokuapp.com" ..... >
  <trk>
    <name>2017-10-27 05:43:05</name>
    <cmt>HMAC Digest: 8d336dc36e1fbd0d1bfdb6f93fbd09ecdabb25f5ce0850438e935d6b04e5fafd</cmt>
    <trkseg>
....
Title: Re: Slim down your bloated GPX track files
Post by: Genosse Brymbo on 28 October, 2017, 07:03:03 pm
As a little extra I thought I'd get it to create a HMAC of its work
What's the hashing algorithm and what data is being hashed?
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 28 October, 2017, 07:32:11 pm
As a little extra I thought I'd get it to create a HMAC of its work
What's the hashing algorithm and what data is being hashed?

SHA-256 and the set of track points belonging to the track are being hashed.
Title: Re: Slim down your bloated GPX track files
Post by: BobCatMax on 08 July, 2018, 08:55:36 pm
This is a wonderful tool!

Does it maintain cuepoints/way points?

Thank you for taking the time to create this. Forgive my ignorance, is there any way of checking the spat out files new number of points?
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 09 July, 2018, 05:22:16 pm
It only touches tracks and track points. So if you have a route or waypoints in the same GPX file, they will come out the other side unchanged. 

You can see the number of track points in things like Basecamp.  I did run some tests at various tolerances to see what the number track points roughly came out at for different distances and settings.  But have long since forgotten what the results were.  I may add a text file into the returned zip file with the stats as that would easy enough to do.  Something for when it finally rains. I have bike riding to be done.
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 09 July, 2018, 05:39:04 pm
As for below 500 points, below 10,000 points I may add these in as tick box options.  The relationship between tolerance and the number of points retained for a distance is surprisingly linear. So it would be relatively easy to get close to the target based on the relationship then a small bit of binary searching to refine.  Again something for a rainy day when the weather finally breaks.
Title: Re: Slim down your bloated GPX track files
Post by: BobCatMax on 10 July, 2018, 06:26:33 am
Fantastic, thanks for your replies.

I won't wish on the rain just yet!
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 23 July, 2018, 02:15:45 pm
Hi,

Updated today, got bored waiting for a rainy day.

Simplify a track down to 500 or 10,000 points

You can now select 500 or 10,000 points as the tolerance in the drop down.

It will aim to get within 1% of the chosen number of points. 

For 500 points it will return a track with between 495 and 500 points.
For 10,000 points it will return a track with between 9,900 and 10,000 points.

If the track is already below the required number of points, it will just return your original track but with the crap cleared out of it. Retain elevation and / or timestamps is respected when filtering for 500 or 10,000 points.

Statistics

In the zip file returned will be a stats.txt file containing statistics on the simplification carried out

Simple GPX
https://simple-gpx.herokuapp.com
Tolerance chosen: 500
Original trackpoints: 4009, simplified trackpoints: 498

Cheers

Phil
Title: Re: Slim down your bloated GPX track files
Post by: JulesP on 23 July, 2018, 02:20:39 pm
You can now select 500 or 10,000 points as the tolerance in the drop down.

What's the rationale for these two vastly different counts? Do they correspond to the ideal number of track points to two particular types of GPS device, or something else..?

I use your excellent tool all the time, but I'm never sure I need to with a Garmin 810 or 1000 (though I've had past problems with RwGPS single GPX files for 200km routes).
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 23 July, 2018, 02:39:04 pm
500 track points is the limit for older GPS units.
10,000 track points is the limit for for GPS units of the generation that saw the Edge 205, Edge 305, Edge 500, Edge 605, Edge 705, Etrex 20, 30 come out etc.  Not sure of the track point limit of Edge 810 but anecdotally I believe it has the same limits.

A 200km track generated online is unlikely to exceed 10,000 points but it has certainly been seen on a 300km track.  I have seen some RWGPS tracks with a trackpoint every 7m even on straight roads.  This is about 28,000 track points for a 200km ride. An unnecessary number. By default I run all the tracks I intend to navigate through my own tool, this includes tracks I have generated myself for DIYs etc.   

But it is not just about these upper limits for tracks.  A clean track with fewer trackpoints is also less likely to glitch the unit.  An example is tracks with timestamp data in them.  If some Edge units encounter timestamps in a track they are trying to navigate then they will glitch / fail.  Etrex are more tolerant than Edge units and less likely to glitch but they will encounter problems if you try and navigate a track above its track point limits.

Certainly since I started only using cleaned / filtered tracks I have not had a problem with glitches / failed navigation.


Title: Re: Slim down your bloated GPX track files
Post by: citoyen on 23 July, 2018, 07:49:50 pm
Top work, Phil, very nice. I especially like having the stats file to compare.

I normally use the 5m tolerance setting, which produces files that are small enough for my Edge 510 to handle. Using the 500pt setting on a 200km route, they're about half the size again. The 500pt tracks seem detailed enough for navigating country roads, although there were a couple of places on my test routes where I would want a few more track points (mainly in towns).

Certainly since I started only using cleaned / filtered tracks I have not had a problem with glitches / failed navigation.

I've had a few problems in the past caused by excessively large GPX files - mainly ones downloaded from RWGPS. Worst example was a ride where the file I was using was so bloated, it took 15 minutes to load. By the time I got going, I was well behind the field, hence when my device crashed less than 20km later, I was without any kind of navigation nor even a wheel to follow. Luckily I knew roughly where the first control was but I still got lost a couple of times and added a good 10km to my distance. Eventually I got it working, luckily, otherwise I'd have had to abandon the ride. I learnt two lessons on that ride: one, always carry a printed routesheet as backup; two, always check that your GPX files work on your device before setting off.

Not that I've had any such problems again since learning how to make files smaller and strip out the bloat.
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 23 July, 2018, 09:11:56 pm
I have never ridden on a 500 point track for a 200km. I normally use the 10m setting which returns around 1100-1500 points depending on how twisty the route.  I have an overnight DIY 200 this Friday to beat the heatwave during the current RRTY run.  I might just try out a 500 point version.  Certainly tracks with less track points and bloat load quickly  and require less processing by the low power GPS CPUs
Title: Re: Slim down your bloated GPX track files
Post by: Ben T on 23 July, 2018, 09:25:58 pm
If I want to reduce to 500 it should reduce to exactly 500, not 498. ;) B minus Phil  ;)
Title: Re: Slim down your bloated GPX track files
Post by: frankly frankie on 24 July, 2018, 11:23:53 am
I normally use the 5m tolerance setting, which produces files that are small enough for my Edge 510 to handle. Using the 500pt setting on a 200km route, they're about half the size again. The 500pt tracks seem detailed enough for navigating country roads, although there were a couple of places on my test routes where I would want a few more track points (mainly in towns).

There was a 400km event in the N-E a couple of years ago where several people were reporting trouble with the organiser-supplied GPX file - I had a look at it out of interest and it was about 13,000 points (sourced RWGPS).  I downsampled it to 500 and although there were a couple of towns where it did look a bit rough, even so every junction in 400km that would have involved a 'turn' was still properly described by the track shape.  An extreme example - obviously in practice the best approach would have been to divide the Track into 3 or 4 and then optionally downsample.
Title: Re: Slim down your bloated GPX track files
Post by: citoyen on 24 July, 2018, 11:51:50 am
I had a look at it out of interest and it was about 13,000 points (sourced RWGPS).

As you've mentioned before, it's not just the number of points but the extraneous data - eg co-ordinates given to microscopic levels of precision. One of the things I like about Phil's app is that it makes files much smaller without necessarily reducing the number of trackpoints significantly. 13,000 trackpoints does seem somewhat excessive though.

You have to wonder why the authors of the RWGPS app don't do something about this themselves, as surely it would be easy enough to fix.

Quote
I downsampled it to 500 and although there were a couple of towns where it did look a bit rough, even so every junction in 400km that would have involved a 'turn' was still properly described by the track shape.

The practicality of 500pt tracks possibly depends on what device you're using - on a device without maps, more trackpoints are beneficial in towns where a junction might offer two or more roads going in approximately the right direction. Without a map for reference, it's sometimes hard to know which one to choose.
Title: Re: Slim down your bloated GPX track files
Post by: frankly frankie on 24 July, 2018, 12:10:27 pm
It's not usually critical in a town.  Unless you're heading for an actual control location, as a randonneur all you need to do is get across the town and out on the right road, there may be 100 ways of doing that with not much time/distance penalty between the best and the worst.

13,000 trackpoints does seem somewhat excessive though.

Excessive as in, most Garmins used by cyclists have a limit of 10,000.  To make it worse, they often won't report a 'too many points' error (or if they do, it's just an irritating little pop-up you click through during boot-up) but instead simply truncate and the first you know about it is in the middle of nowhere.  That makes either Garmin or RWGPS or both somewhat liable for a situation that could conceivably be worse than just a bit uncomfortable.
Title: Re: Slim down your bloated GPX track files
Post by: JulesP on 24 July, 2018, 03:47:18 pm
To make it worse, they often won't report a 'too many points' error (or if they do, it's just an irritating little pop-up you click through during boot-up) but instead simply truncate and the first you know about it is in the middle of nowhere.

That's exactly what happened to me on a RwGPS-plotted 200 route on a Garmin 810. At around 130km, it suddenly switched to a straight line that went who-knows-where (the destination, presumably). This was before I knew of such glitches but, after following someone to the next control and then restarting the route, all was well.

It's not happened since I started using Phil's tool, nor since I started splitting files to ~100km/inter-control tracks (the latter using a paid RwGPS account).
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 24 July, 2018, 03:56:52 pm
I have reversed the order it will split and simplify a track.

So (if requested) it will split the track first and then it will simplify each track.  Makes no odds if using the metre tolerances.  But if using the 500 point or 10,000 point tolerances and splitting as well you will get a different result. 

Let's say you decide to split a PBP GPX track into lengths of 300km with a tolerance of 500 points.  Then you will get 5 tracks (PBP is longer than 1200km), and each track will have 500 or less points. Under the old method it would have reduced the original 1200km track to 500 points and then split it, ending up will quite a bit less points in each track.   

The stats.txt file returned has been updated to include track splitting data. Below are figures for the WAWA 2016 2100km GPX track split into 200km chunks.

Simple GPX
https://simple-gpx.herokuapp.com
Tolerance chosen: 500
Split track: yes, split distance: 200km
Original tracks: 1, simplified tracks: 11
Original trackpoints: 21734, simplified trackpoints: 5390, average trackpoints per track: 490
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 24 July, 2018, 04:27:00 pm

It's not happened since I started using Phil's tool, nor since I started splitting files to ~100km/inter-control tracks (the latter using a paid RwGPS account).

I have thought about adding the option to split tracks at controls but it only works if

The GPX file contains waypoints for every control and the control waypoints are within a reasonable proximity of the actual track (say within 50m).  You do not have extra waypoints in the GPX file being used for other purposes. 

I am not a fan of splitting tracks at controls as it gives me something extra to do when there is plenty to do at the control anyway.  Generally I run long tracks and only split them where they cross over themselves. It is the number of points causing truncation not the length of the track in km.

But certainly adding split by control would not be that hard with the above provisos.
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 25 July, 2018, 04:11:36 pm
If there are no routes in the GPX then additionally it will now.

Generate a route from the track, with an average of one route point for every 5km.  Obviously the spread of route points will vary, with more where twisty and less when not. If it finds waypoints (it assumes they are controls / places you wish to pass through) then it will try and insert them into the route in the appropriate place.  It will then split the route into 50 route point chunks so it can be auto routed in GPS that support it.

Below is one generated through the process with flags representing via route points.  Use at your own risk etc. etc. or just stick with the simplified tracks which it already outputs.

(https://www.dropbox.com/s/yktafu01x6g8so3/Capture.JPG?raw=1)



Title: Re: Slim down your bloated GPX track files
Post by: frankly frankie on 25 July, 2018, 06:35:37 pm
Possibly this (and/or possibly the stats.txt file) might break the zip file for the purpose in post #1 of this thread, that is, submission to AUK and subsequently running it through the RouteValidator tool.  I don't know that it does - just speculating, I know that RV did have some difficulties initially with zip files having unexpected contents, and I know that was fixed to some extent but I don't know how bomb-proof the fix is.

However this project has maybe gone way beyond that, and from that angle, it's all good  :thumbsup:

edit: no I've tested this now in RV, and it still seems to work OK, ignoring both the added Route and the added stats.txt file.
Title: Re: Slim down your bloated GPX track files
Post by: citoyen on 26 July, 2018, 02:59:29 pm
Phil, I've tried to use the app a couple of times this morning and got an application error. Problem at my end or yours?

Slightly OT question regarding routes: my device doesn't support routes so they're extraneous data for me personally. However, is it worth keeping them in the GPX files that I supply to riders for use on my events? What is the benefit of routes if your GPX file also contains a track?
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 26 July, 2018, 06:25:46 pm
Some prefer tracks, others prefer routes for the turn by turn, some like to use both.  I generally have't used routes as my experience has been that they did not go where I wanted them. When creating routes I do not click every junction. So when it gets to the gps it rarely looked like what I had in Basecamp. But with a route filtered from a track it seems from what I can see that Basecamp and my gps concur.  So I may just revisit routes again for my own nav though I am quite used to tracks. I would leave the route in the GPX as an optional extra!

Regards other point will see if I can see in logs what it did not like about your GPX
Title: Re: Slim down your bloated GPX track files
Post by: citoyen on 26 July, 2018, 06:32:05 pm
Thanks, Phil.

Just been reading up on routes, in relation to the other thread. I think I get the general idea now. It seems to me that for following a prescribed route (as on an audax), tracks are probably a better option.
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 26 July, 2018, 07:07:13 pm
The only thing I can see in the logs are timeouts at 8:51 and 9:06am.  Was that when you were using Simple GPX?   Is it a GPX with a particularly large number of track points or waypoints and what settings were you using? Mind I have run it against a track with 21,000 track points without issue.
Title: Re: Slim down your bloated GPX track files
Post by: citoyen on 26 July, 2018, 10:48:51 pm
The only thing I can see in the logs are timeouts at 8:51 and 9:06am.  Was that when you were using Simple GPX?   Is it a GPX with a particularly large number of track points or waypoints and what settings were you using? Mind I have run it against a track with 21,000 track points without issue.

Sounds about right, time-wise. If the problem is timing out, that could be due to my flaky internet connection.
Title: Re: Slim down your bloated GPX track files
Post by: citoyen on 27 July, 2018, 08:54:06 am
Forgot to add: it was an RWGPS file, 585KB, 7161 points, set to 5m tolerance
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 27 July, 2018, 10:39:28 am
Your GPX works fine now.  Trying to hit too fine a points tolerance when generating a route from such a short track meant it hit the timeouts.  I have relaxed the tolerances for route generation.

Route generation is now also a seperate action in the tolerance drop down rather than an automatic operation every time.  If you now select generate route in the dropdown then that is all it will do, leaving the original track alone.



Title: Re: Slim down your bloated GPX track files
Post by: citoyen on 27 July, 2018, 10:49:32 am
Top work, Phil. :thumbsup:
Title: Re: Slim down your bloated GPX track files
Post by: frankly frankie on 27 July, 2018, 11:10:07 am
Very good.  I would only comment that if you now want a Track and Route, you end up with two GPX files of the same name.  It would be nice (I know  ::-)) if the route filename were suffixed -r or similar?
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 27 July, 2018, 11:27:15 am
There you go it will now add   _route to the end of the zip file and GPX names if you are generating a route. Be aware the original track file will still be in a GPX where you have generated a route.  Could not decide whether to leave the original track in there or delete it when generating a route.
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 27 July, 2018, 11:36:11 am
Strike that, if generationg a route the returned GPX will only contain the route and any original waypoints. It will now delete the original track.  That way you will not accidently load an unfiltered bloatware track if you load the GPX route file.
Title: Re: Slim down your bloated GPX track files
Post by: frankly frankie on 27 July, 2018, 05:56:14 pm
Very useful utility.  :thumbsup:
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 28 July, 2018, 02:36:11 pm
Thanks I cannot see me adding anything more to it unless someone has a killer idea. The Heroku platform and plan I have  is free for life but should they ever shut down I can shunt the application across to other servers I run stuff on.
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 13 August, 2018, 05:34:50 pm
So much for not adding anything more.

It can now split tracks by control.  It is a new option in the split track dropdown.  Track naming works the same way as split by distance.  It looks for waypoints in the GPX, finds the nearest trackpoint for each one. Then splits the track accordingly.  It ignores any waypoints close to the start or end of the original track; to stop any stupidly short tracks being created.
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 25 September, 2018, 04:34:00 pm
It can now add proximity alarms to all the waypoints in a GPX file. New dropdown to select from 50 metres to 400 meters.

https://simple-gpx.herokuapp.com/

Title: Re: Slim down your bloated GPX track files
Post by: De Sisti on 25 September, 2018, 05:38:59 pm
Slightly OT, but I find that when my Garmin Edge Touring is connected to my computer
I can drop any gpx (or tcx) file that I have on my machine into the Touring's 'NewFile'
folder. The gpx/tcx are then automatically converted into a .fit file (which the unit prefers)
and moved to the 'Courses' folder.

It works on some of the smaller rides I've done recently, but will try it on a long ride
soon (which will be the Upper Thames 200 in November).
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 17 June, 2019, 02:03:42 pm
Will now split out waypoints, tracks, routes into seperate GPX within the Zip file.  If you have multiple tracks or routes they will each be within their own GPX file.  Naming of GPX files is self evident hopefully.
Title: Re: Slim down your bloated GPX track files
Post by: JulesP on 17 June, 2019, 09:07:10 pm
I'm now getting "undefined0" appended to downloaded file names, but the contents appear to be OK. Just an FYI.
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 17 June, 2019, 11:12:18 pm
I'm now getting "undefined0" appended to downloaded file names, but the contents appear to be OK. Just an FYI.

Ah, I half started making some changes with the GPX file names then changed my mind. Probably left a bit of that in for certain actions. That won't affect contents. Will sort tomorrow.
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 18 June, 2019, 12:36:36 pm
Sorted
Title: Re: Slim down your bloated GPX track files
Post by: el on 01 November, 2019, 06:09:17 pm
Hi- I'm new to this forum.
Do I understand correctly that if I put gpx files through this website to be made smaller, my Garmin 800 will no longer crash leaving me to rely on my paper cue sheets or other participants?
Title: Re: Slim down your bloated GPX track files
Post by: Phil W on 01 November, 2019, 08:14:25 pm
Hi- I'm new to this forum.
Do I understand correctly that if I put gpx files through this website to be made smaller, my Garmin 800 will no longer crash leaving me to rely on my paper cue sheets or other participants?

No guarantees about the quirks of your Garmin Edge 800 but it'll certainly reduce the frequency / likelihood of it crashing when navigating the GPX. Note it slims GPX tracks only, not GPX  routes which are a different beast.
Title: Re: Slim down your bloated GPX track files
Post by: fhills on 01 November, 2019, 09:37:46 pm

No guarantees about the quirks of your Garmin Edge 800 but it'll certainly reduce the frequency / likelihood of it crashing when navigating the GPX. Note it slims GPX tracks only, not GPX  routes which are a different beast.
They aren't much different at all I don't think if you use point to point navigation, riding between marks of the turns.

I slim down GPX tracks/routes semi manually for use on an etrex.
Get gpsies.com to reduce the points automatically if needed (not really needed with my own routes from cycletravel as they only mark turns to begin with) then fine tune on the gpsies map, then semi automatically strip stuff out using a simple text editor.
There's an awful lot of not needed junk in most gpx files.
Title: Re: Slim down your bloated GPX track files
Post by: grams on 01 November, 2019, 09:46:59 pm
Hi- I'm new to this forum.
Do I understand correctly that if I put gpx files through this website to be made smaller, my Garmin 800 will no longer crash leaving me to rely on my paper cue sheets or other participants?

I believe the only way to stop a Garmin Edge of that era crashing is the War Games solution.
Title: Re: Slim down your bloated GPX track files
Post by: Captain Offensive on 23 August, 2021, 09:59:06 am
Hi Phil,

Just wanted to say that I just came across this thread and your webpage and just wanted to say thanks.  I've been adding some GPX files to my blog using a great new free mapping tool/Wordpress plugin from the OS.  My GPX files were all a bit large though so I've been compressing them down using your tool just using 1m accuracy and its worked a treat. For instance the GPX file that I had on page https://www.richmay.co.uk/apn4/ has gone from 22Mb to 500Kb without losing any accuracy on the mapping page and making loading time of the page much better.

Cheers
Rich
Title: Re: Slim down your bloated GPX track files
Post by: Pingu on 23 August, 2021, 10:42:22 am
22Mb  :o ???
Title: Re: Slim down your bloated GPX track files
Post by: Lightning Phil on 05 September, 2021, 01:40:14 pm
I leave it at default 10m accuracy for my audax rides. I doubt you’d tell difference with that either. You’d have to be zoomed right in so that 10m was represented by multiple pixels on a screen.  Even then you are down to just not showing a perfect curve round a mini roundabout.  Which is seriously zoomed in.  Possible on computer screen but on much smaller GPS screen not very likely.
Title: Re: Slim down your bloated GPX track files
Post by: αdαmsκι on 17 November, 2021, 09:18:00 am
Normally works great but this morning I am getting "Internal Server Error".
Title: Re: Slim down your bloated GPX track files
Post by: Lightning Phil on 17 November, 2021, 02:39:42 pm
Just tried it and it’s working fine with the GPX I just tried.  The hosting is free so don’t know if they had problems this morning.
Title: Re: Slim down your bloated GPX track files
Post by: αdαmsκι on 17 November, 2021, 03:14:59 pm
May explain the issue; anyway now working here too. Thanks Phil.
Title: Re: Slim down your bloated GPX track files
Post by: Lightning Phil on 17 November, 2021, 06:26:22 pm
Heroku on which it runs is something called platform as a service.  This means I don’t have anything to do with the servers or their settings.  I just load my code to a git repository when I have updates and their platform does the magic to deploy it.  I can see the logs but seeing as my code is fairly static, not changing all that often, any problems are likely their end.

It’s a free service at the level I use it but Salesforce have bought Heroku. So we will see how long that lasts. I’ll only keep it up there whilst it’s free. I might experiment turning it into a desktop app over the winter. Maybe.  My enthusiasm for coding stuff waxes and wanes.
Title: Re: Slim down your bloated GPX track files
Post by: Lightning Phil on 25 April, 2022, 03:42:41 pm
Have added two new features and amended proximity alarms in latest release.

(https://lh3.googleusercontent.com/pxeXEP03MRnc9Yixj85cYLIycZal31ICjgNzCvVFNlbHyCYKfWjEEOWBlWVhaAbtoBY1XXX3Qgw_evjTo0b5HBdyP-TIG3PNPsgggX2hKdLsEwP_H5M320paDavwrAQ7W8B8hSUtLw=w2400)

Join Tracklogs

You can now join tracklogs together by simply selecting multiple GPX files containing tracklogs.  It does the rest by looking at the start timestamps of each tracklog segment, and sequencing them in chronological order.  Tracklog segments are kept intact and a single GPX tracklog is returned in a zipped file. The GPX and zip files are named according to the earliest timestamp found.  None of the simplification steps are applied during this operation.  You'd need to load up the resulting single tracklog in a second step to apply simplification to it, to reduce its size etc, if an online service objects to overly large tracklog files.

Something I'll be using myself on my 400km audax next month, where I normally auto split tracklogs at midnight, and rejoin them after the event.

Distance Based Waypoint Alarms

You can now add distance based waypoints with proximity alarms for supplied tracks.   The waypoints occur at your specified distance interval, with the prefix you choose, with a suffix that indicates the distance the waypoint occurs at. I give them the red pin 📍 symbol which is recognised by Garmin software but may not be recognised by other software.

Proximity Alarm Radius

You can now set the proximity alarm radius (for existing waypoints) to 25m after  a request for that option.

Under The Covers

I've now changed it to run in a Docker container as Heroku's version of NodeJS was getting long in the tooth.  Now runs on Alpine Linux 3.15.1 and NodeJS 17. Also more easily moved should I need to move away from Heroku.
Title: Re: Slim down your bloated GPX track files
Post by: citoyen on 25 April, 2022, 04:20:23 pm
Nice idea with the joining tracklogs.  :thumbsup:

I've done this manually in a text editor, but letting a machine do it for me sounds like a good way to eliminate the potential for human error.
Title: Re: Slim down your bloated GPX track files
Post by: aidan.f on 28 April, 2022, 07:03:59 am
Phil,

Thank you for updating the waypoint alarm to 25 m at my  request  - Will be testing the result on Deans  'Highland  Fling' 1000K

Quote
It’s a free service at the level I use it but Salesforce have bought Heroku. So we will see how long that lasts. I’ll only keep it up there whilst it’s free. I might experiment turning it into a desktop app over the winter. Maybe.  My enthusiasm for coding stuff waxes and wanes.

I do like it the way it is  - machine agnostic. Ues it lots. If it started to cost I would be happy to part fund..
Title: Re: Slim down your bloated GPX track files
Post by: Lightning Phil on 28 April, 2022, 10:12:53 am
The free package hasn’t changed so far, and simple-gpx and its use case fits easily within the specifications and terms and conditions of free.  Hopefully there’s enough income from the paid Heroku packages that they keep the free option ticking along.

Good luck with Highland Fling 1000 this weekend.
Title: Re: Slim down your bloated GPX track files
Post by: Lightning Phil on 04 May, 2022, 02:55:10 pm
,
Thank you for updating the waypoint alarm to 25 m at my  request  - Will be testing the result on Deans  'Highland  Fling' 1000K

I see you finished, how did the 25m proximity alarms work out?
Title: Re: Slim down your bloated GPX track files
Post by: aidan.f on 23 May, 2022, 06:23:57 am
Better late reply than never, the ride was good  alarms not so good I am going to have to do some work optimising waypoint locations in rwgps for my etrex.
Title: Re: Slim down your bloated GPX track files
Post by: Lightning Phil on 26 August, 2022, 09:28:40 am
Well free hosting is ending at Heroku but not till November.   Thus Simple-GPX will cease to exist online from November, unless I find free hosting in the mean time.
Title: Re: Slim down your bloated GPX track files
Post by: frankly frankie on 26 August, 2022, 10:40:52 am
If it started to cost I would be happy to part fund..

+1 (me too) :thumbsup:
Title: Re: Slim down your bloated GPX track files
Post by: psyclist on 26 August, 2022, 01:29:32 pm
Well free hosting is ending at Heroku but not till November.   Thus Simple-GPX will cease to exist online from November, unless I find free hosting in the mean time.

Oh no, just as I've started to use it as part of my workflow when submitting DIY rides.

Would it take much to package it up as an app that could be run locally, if that was an option you would consider?
Title: Re: Slim down your bloated GPX track files
Post by: Lightning Phil on 26 August, 2022, 07:12:41 pm
I’ve got 3 months before they charge.


I had a look around at cloud offerings this afternoon.   It looks like it’ll fit within the free tier option on Google Kubernetes.   It’s packaged in a Docker container , I should just need to set it up in Google console then deploy the container to Google. Then find out what url it has been assigned.

I’ll take a look at doing that September once I’ve confirmed my understanding is correct on the free tier conditions
Title: Re: Slim down your bloated GPX track files
Post by: chrisbainbridge on 02 September, 2022, 03:56:38 pm
it is giving an error message at the moment
Title: Re: Slim down your bloated GPX track files
Post by: chrisbainbridge on 03 September, 2022, 10:30:12 am
Working again
Title: Re: Slim down your bloated GPX track files
Post by: psyclist on 30 November, 2022, 07:02:55 am
The free package hasn’t changed so far, and simple-gpx and its use case fits easily within the specifications and terms and conditions of free.  Hopefully there’s enough income from the paid Heroku packages that they keep the free option ticking along.

Has the free Heroku package ceased? No sign of life for the little simplify GPX app this morning.
Title: Re: Slim down your bloated GPX track files
Post by: Lightning Phil on 30 November, 2022, 06:48:39 pm
Bugger, ended on Monday. Forgot about that. I’ll take a look over the coming week at the alternate free hosting I was looking at.
Title: Re: Slim down your bloated GPX track files
Post by: freeflow on 28 December, 2022, 04:23:15 pm
Site is not responding today.
Title: Re: Slim down your bloated GPX track files
Post by: Lightning Phil on 28 December, 2022, 04:25:26 pm
It won’t till I move it to a new free hosting site.  I’ll take a look at doing that Friday. Need to be in front of PC to do that.
Title: Re: Slim down your bloated GPX track files
Post by: Lightning Phil on 28 December, 2022, 07:22:17 pm
Okay now running on Google Cloud and should be free for the amount of use it gets (I hope). Replace the old url with this one.

https://simple-gpx-ncmlmbodrq-ew.a.run.app/
Title: Re: Slim down your bloated GPX track files
Post by: psyclist on 28 December, 2022, 07:31:48 pm
Okay now running on Google Cloud and should be free for the amount of use it gets (I hope). Replace the old url with this one.

https://simple-gpx-ncmlmbodrq-ew.a.run.app/

Great news, thank you. I've never had much success finding anything quite as straightforward to use for getting GPXs down to a reasonable size for DIY and ECE validation.
Title: Re: Slim down your bloated GPX track files
Post by: D.A.L.E. on 28 December, 2022, 07:48:13 pm
Thanks Phil, it's been a great little tool over the years. If you ever need it, I'd be happy to donate.
Title: Re: Slim down your bloated GPX track files
Post by: Lightning Phil on 30 December, 2022, 03:02:00 pm
Thanks, looks like I incurred £1.75 since setting up couple of days ago.    But I think that was just a misunderstanding on my part of the all the components it needs from Google Cloud.  Why can't these cloud providers use the same names for stuff? I've deleted / removed some Google services that I've realised weren't needed and generated those costs, and it still seems to be up and running. Looks like you get 50 mins of CPU free per month (on the Google Cloud service it is now running on), which should be enough for the usage it gets (I think).  I'll need to keep an eye on it for this first month, so no surprise bills. 

Title: Re: Slim down your bloated GPX track files
Post by: willpom gwraudax on 05 January, 2023, 12:06:37 pm
Thanks, looks like I incurred £1.75 since setting up couple of days ago.    But I think that was just a misunderstanding on my part of the all the components it needs from Google Cloud.  Why can't these cloud providers use the same names for stuff? I've deleted / removed some Google services that I've realised weren't needed and generated those costs, and it still seems to be up and running. Looks like you get 50 mins of CPU free per month (on the Google Cloud service it is now running on), which should be enough for the usage it gets (I think).  I'll need to keep an eye on it for this first month, so no surprise bills. 

Thanks Phil, this is a great tool. I've DM'd you.
Title: Re: Slim down your bloated GPX track files
Post by: Lightning Phil on 20 February, 2023, 12:48:08 pm
I've been working on the ability for simple-gpx to take a GPX with tracks and waypoints in, and as well as being able to add waypoint proximity alarms, it can also convert those waypoints to work as cue points recognised by Garmin Edge and Wahoo GPS. 

It will do several things


The proximity alarm drop down will now have an option Garmin Edge/Wahoo cue point.   It will also add the elements required for cue points even if you selected a proximity alarm as the etrex ignores the cue point elements.

To help my workflows as an Audax organiser I'm now going to include the waypoints in the output track GPX.   This will enable those to be imported into Garmin Connect and Ride With GPS with the control cue points intact, to allow easy syncing with Garmin Edge and Wahoo GPS for those riding your events.  I'll still output the waypoints in a separate GPX for anyone who wants them separate.  This should enable a single clean workflow for providing the control alerts whether Garmin Etrex, Edge or Wahoo

It's in testing at the moment with tame Wahoo users (I now own both an Edge and Etrex GPS so can test those bits myself).

(https://lh3.googleusercontent.com/MM-NfC2A69JoqAoKeUzUHR6GXJvdtWYbOW9r5ThK1FxE0tjKTj5OJpj-t3DXsSdgX38LrFM44BD65D9AgKq0LlJLvExPLDLGXAVqkF3qosAT-UEWThtMcd850n_hfgdFRZ_QSqkwDg=w2400)



Title: Re: Slim down your bloated GPX track files
Post by: madcow on 16 April, 2023, 09:19:37 pm
Just here to publicly thank you Phil.
I use a Garmin Dakota 20 because it runs off AA batteries, does the job and I'm not spending ££££s to uprate .
I had a problem with a track and someone suggested using software like yours.
I wish that I had read this thread before my 300 k rather than after. :facepalm:

BTW it was the  Gainsborough to Alfreton section where my track disappeared .
I've done it so many times that I was calling the turns before my companions Garmin was.
Title: Re: Slim down your bloated GPX track files
Post by: Lightning Phil on 17 April, 2023, 10:28:35 pm
Thanks mad cow. 

I’m working on a revamped version,  with additional functionality, now that Basecamp just crashes on my PC.  But no timescales as of yet, early days.