Author Topic: PBP Garmin File Upload Issue - Can anyone help?  (Read 18171 times)

Zed43

  • prefers UK hills over Dutch mountains
Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #25 on: 25 August, 2019, 11:20:35 am »
The full .fit file is 11MB, still well under Strava's limit of 25MB. When I filter out the HRV data it comes down to 7MB, but still no go.

Just curious for those of you who also have problems:
* do you have a power meter
* do you have Di2 (or eTap)
(I'm a "yes" to both)

Extremely annoying, this. But what really vexes me is the lack of response from StravaSupport; why the heck do I pay for the full Summit package if they can't even respond in person to a problem (it's been two days now without response...)

Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #26 on: 25 August, 2019, 11:20:43 am »


.fit files seem to be pretty damn efficient then. Just noticed the .fit file for last weekend's 300 is 1.8M, where as the GPX is 14M.

I wonder what arbitrary dice roll strava used to decide on the upload limit...

J

They must be cos I set my data intervals quite narrow, got heart rate and power meter connected and uploaded as well.

Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #27 on: 25 August, 2019, 11:23:35 am »
* do you have a power meter
* do you have Di2 (or eTap)

nope. just heart rate data. the two fit files I am trying to combine and upload are 7.5mb and 700kb.

fitfiletools also seems to have broken on me now "Error: Java heap space"

 ???

Zed43

  • prefers UK hills over Dutch mountains
Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #28 on: 25 August, 2019, 11:25:46 am »
GPX files are uncompressed XML, that is always going to be big (it compresses well though).

@Hofnar with your workaround you get two (or three) separate rides in Strava I suppose? It doesn't get merged into one 1200km ride?

simonp

Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #29 on: 25 August, 2019, 11:32:38 am »
It's not that .fit is efficient, it's that gpx is an incredibility inefficient format. The extensions make it more so.

quixoticgeek

  • Mostly Harmless
Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #30 on: 25 August, 2019, 11:36:39 am »
.fit files seem to be pretty damn efficient then. Just noticed the .fit file for last weekend's 300 is 1.8M, where as the GPX is 14M.

Surely there’s no need for the GPX file to be anything like that big, even for a 1200km ride? Is it not possible to run it through eg Phil W’s simplegpx tool to strip out extraneous data?

Oh god, this again.

Probably. I hit "download gpx" on strava, and looked at the size it produced. The gpx file exported from strava, is derived from the .fit file that went in from my wahoo. Thus as a comparison of efficiency of .fit vs .gpx, running it through anything would null the comparison.

Then there is the issue that disk space is cheap, I'm approaching the point where the amount of disk storage I have in the house is measured in petabytes. So a 14meg file is not worth my time and effort to make it smaller.

Does the GPX file need to be that big? probably not. Is it a problem that it's that big? not at all.

J
--
Beer, bikes, and backpacking
http://b.42q.eu/

LittleWheelsandBig

  • Whimsy Rider
Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #31 on: 25 August, 2019, 11:48:09 am »
Except when it is...
Wheel meet again, don't know where, don't know when...

simonp

Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #32 on: 25 August, 2019, 11:50:35 am »
Often there's redundant whitespace in a .gpx which inflates the size. However the size of individual datapoints is at least several times, probably more, what they would be in a binary representation. The only advantage .gpx (and .tcx) has is it's human readable and thus possible to fix-up issues by hand.

Strava appears to be broken anyway - it can't process the large .fit file but it gets stuck in limbo so you are left in an unfixable mess. And their support is terrible.

citoyen

  • Occasionally rides a bike
Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #33 on: 25 August, 2019, 12:01:34 pm »
Except when it is...

Exactly.

Though it sounds like file size is not actually the problem here.
"The future's all yours, you lousy bicycles."

zigzag

  • unfuckwithable
Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #34 on: 25 August, 2019, 12:06:45 pm »
fwiw, my .fit file that includes hr, cadence and power is 6.0mb for the whole pbp ride. (strava file size limit 25mb, so good for up to ~5000km. i've uploaded 4000km tcr ride without a problem before).


Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #35 on: 25 August, 2019, 12:35:25 pm »
Solution gained.

I posted a message and request for help on the Garmin Forum and a very kind person of the name 'OnlyTwo' had a look and re-sized the file for me (10Mb down to 1.3Mb) and it has uploaded to Strava with no issues. I have no idea what 'OnlyTwo' did, but I am very grateful to them.

Sorry that so many of us seem to be having similar issues.

Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #36 on: 25 August, 2019, 01:06:39 pm »
Strava uses AWS, at a guess imports are handled by a lambda instance or similar. My guess is if processing does not complete by a timeout the process hangs in this middle ground.

This would explain my fix if making file much smaller and offsetting time to make it a 'new' ride.

although the is a file size limit

Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #37 on: 25 August, 2019, 01:19:01 pm »
Since Garmin Connect does not support my 'dumb' etrex-30, I have just joined the tracks together (all three of them for PBP) and uploaded them to starve as GPX files without any issues. This does require a USB cabled connection to a computer to do this; thankfully a mate had his laptop at the finish and we did this over a beer  :thumbsup: :thumbsup:

simonp

Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #38 on: 25 August, 2019, 01:56:41 pm »
etrex 30 will not record so much data, and might record fewer samples - a much smaller gpx will result.

My Summit membership is up for renewal next month, I noticed.

They can't handle the file - ok. They leave it in limbo so you have to jump through hoops to try to sort it out - nok.

Kim

  • Timelord
    • Fediverse
Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #39 on: 25 August, 2019, 03:24:12 pm »
I wonder what arbitrary dice roll strava used to decide on the upload limit...

PHP default memory limit? :)


Strava's rejection of duplicates is a royal pain in the arse.  Particularly if you want to upload a file and crop different parts out of it (those of us with sensible Garmins end up with two rides in the same file when we forget to press the right button at the start of a ride, rather than no data).

Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #40 on: 25 August, 2019, 03:32:59 pm »
GPX files are uncompressed XML, that is always going to be big (it compresses well though).

@Hofnar with your workaround you get two (or three) separate rides in Strava I suppose? It doesn't get merged into one 1200km ride?

Two or three separate rides sadly.
My main interesst is Power data, flyby and some interesting segements so its not that bad for me. Would be nice to get on ride but there for you need to compress data and that's past my skills.

Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #41 on: 25 August, 2019, 04:03:54 pm »
Solution gained.

I posted a message and request for help on the Garmin Forum and a very kind person of the name 'OnlyTwo' had a look and re-sized the file for me (10Mb down to 1.3Mb) and it has uploaded to Strava with no issues. I have no idea what 'OnlyTwo' did, but I am very grateful to them.

Sorry that so many of us seem to be having similar issues.

I asked what 'OnlyTwo' did and they responded with the following;

How did you fix it?
  • used FitFileRepairTool
  • import as large file
  • only keep data every 5th second, drop rr and cycling dynamics

    I think that the Fit File Repair tool is a tool that can be downloaded for PC's (at a cost), see below;
    http://fitfilerepairtool.info/

jiberjaber

  • ... Fancy Pants \o/ ...
  • ACME S&M^2
Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #42 on: 25 August, 2019, 06:55:33 pm »
I have that so will try when I get home tonight
Regards,

Joergen

Zed43

  • prefers UK hills over Dutch mountains
Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #43 on: 25 August, 2019, 09:26:36 pm »
I've managed to upload my ride as well. Wrote a program to condense the data (roughly the same as what OnlyTwo did: throw out the HRV data and accumulate HR/speed/distance/power/cadence/temperature but nothing else for 3 seconds into one).

Without any optimisation the (single threaded) program runs in 10 seconds on a MacBook, boggles the mind why Strava doesn't do such a cleanup for large files before further processing.

jiberjaber

  • ... Fancy Pants \o/ ...
  • ACME S&M^2
Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #44 on: 26 August, 2019, 12:45:29 am »
Finally uploaded, I too used fit file repair tool and appended my files whilst downsampling to 5 second and removing power & R-R data.
Regards,

Joergen

Zed43

  • prefers UK hills over Dutch mountains
Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #45 on: 26 August, 2019, 08:42:04 am »
I must be dense this morning, but what is R-R data?

Feanor

  • It's mostly downhill from here.
Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #46 on: 26 August, 2019, 08:57:05 am »
Mine manually uploaded to Strava just fine, it took just a few seconds.
I didn't pass it up via Garmin Connect like I usually do, I wanted to remove as many potential points of failure as I could.

I spliced 4 x 300k files together using the Fit File Repair Tool, did not downsample in any way.
The files only had position / time data; I wasn't recording HR or Power.

Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #47 on: 26 August, 2019, 09:07:15 am »
I must be dense this morning, but what is R-R data?

I wasn't sure either, but googling it shows this from the Polar website; RR recording saves RR intervals, i.e. time between successive heartbeats, for further analysis. Heart rate varies with every heartbeat. Heart rate variability (HRV) is the variation of RR intervals.

zigzag

  • unfuckwithable
Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #48 on: 26 August, 2019, 11:40:51 am »
i've tried the online fit file repair tools, however they did not fix the file in a way that could be uploaded to strava. still have a strava ticket open, but no response from them (except the default automated one). luckily i can see all the metrics in ridewithgps, which is helpful, but i'd still want the file on strava as that is where i log all my riding that matters.

Zed43

  • prefers UK hills over Dutch mountains
Re: PBP Garmin File Upload Issue - Can anyone help?
« Reply #49 on: 26 August, 2019, 12:41:26 pm »
If you have trouble uploading, pm me and I'll give you my email so I can try to condense your file with my program (it's rather user-unfriendly at the moment).