AUK Finances and Website Project was: AUK Chairman Statement

[Jaded]

As this thread is about AUK Finances as well as the website, how much is left in the pot for the recovery of this project?

[frankly frankie]

Irrespective of the grand plans by AUK this work (a+b) needed/needs to be done anyway.

What - and then drive it to the tip?   

Anyway, we'll see what lands in April and what decision the board make between now and then. As others have said I hope they take this as an opportunity to reassess the position.

The thing is that by then there is a tipping point (Phase 2 is a partial data migration) beyond which going back becomes much less realistic.

All AUK actually needs is a correspondant accredited to ACP and a few volunteers to run events.  Other stuff is nice but not essential.

As all aspects of AUK's operation continue to increase (membership, events, rides ridden) the total workload to administer all this is considerable. 
aukweb is all about distributing this work as thinly as possible (members self-administer, organisers self-administer, riders initiate their own recording when they enter online).  This is not the public website (now mostly transferred to a.uk), this is the stuff below the waterline.

The opposite approach which many organisations prefer, is to centralise the work and employ a couple of full-time full-paid administrators.

[Greenbank]

As this thread is about AUK Finances as well as the website, how much is left in the pot for the recovery of this project?

Reports from the 2020 AGM haven't been published yet (it was only on 8th Feb).

Last set of reports are here: https://www.audax.uk/media/2092/bmm-191009-papers.zip

"
Management Accounts

NA reported that income had risen by £23k compared to the same period last year.

The timing of Arrivee and the IT expenditure meant that cash flow was more positive than anticipated but this would reverse as those expenses were realised.

AUK had made a loss of £12k for the full year.  This reflected the ongoing IT costs and the payment of the deposit for Phase 2.  Without that deposit payment there would have been a surplus of around £40k.

The outturn of NA’s budget forecast was within £1k so he was satisfied that the budgeting process was sufficiently robust.

There is currently £210k in the bank.

NA believes the increases made to fees and memberships last year were at the correct level and did not see any need to increase these again this year.

He proposed to talk about the Finances to members at the Reunion as part of the Open Forum
"

The balance sheet shows ~£210k in the bank and £60k of liabilities+memberships leaving ~£145k.

IT spend was £127,787 last year and currently £87,359 this year. No idea what state of deposits/etc for future work this includes.

[Greenbank]

Irrespective of the grand plans by AUK this work (a+b) needed/needs to be done anyway.

What - and then drive it to the tip?   

Anyway, we'll see what lands in April and what decision the board make between now and then. As others have said I hope they take this as an opportunity to reassess the position.

The thing is that by then there is a tipping point (Phase 2 is a partial data migration) beyond which going back becomes much less realistic.

How likely is this to happen given F1/IIP going into administration?

Given that, even if it does, "...the entries will still be processed on the old site, just as happens at present." the existing server might need to keep going for longer than expected.

I look forward to seeing the "Phase 3 requirements document" mentioned in the October 2019 AUK System Refresh update. Hopefully it will be made public.

[grams]

My understanding is that phase 2 migrates the members database to the new platform, but leaves all aspects of events and results management (and entry) on the old server.

Logically Phase 3 should deal with the latter, but in the various past documents I've seen it's a grab bag of wish list items that do anything but.

[Genosse Brymbo]

I'm very impressed with Greenbank's grasp of the issues and his clear analysis.  Everything he says in reply #762 is spot on, from a technical point of view.

Reading this thread while waiting for the legacy code I maintain for a living to build, I'd agree with this.  After reading #762 my immediate reaction was that you could make the changes incrementally and that a division of labour between a 3rd party building some sort of JavaScript web application and others building a backend returning JSON could work.  I note that in #773 Greenbank has further developed his "argument" and added more detail (presumably to some degree in response to FifeingEejit's dive into technical details) to suggest such a division of responsibility.  (I also note he talks of JSON and does't mention "Web api" or "REST api"; I assume in order not to provoke a software theology debate.)

That Greenbank he speak much sense.

[FifeingEejit]

As for whether an insitu redevelopment was realistic, I don't have the info from the discussions or the tech details.
But theres only so much you can do on the same stack, I've heard "PHP has totally changed give it another shot" but it sounds like that means a new website with little portability of existing code, and if there's no separation of concerns or seams in the existing code (which given the code bases age I expect is the case) in the code you're starting from scratch anyway.

My working assumption for in situ redevelopment is that the code that exists *works* and as much of it as possible should be left alone. The things I've heard to be wrong with it could be solved with a few days of strategic search and replace.

It would be nice to have a new tidied schema, but not if it ends up costing you a million quid, which is where this project is heading.

(and AFAIK all new code developed so far is built on the old schema, so all that code is *already* legacy)

This sounds a lot like what management at work thought about our current systems.
Though this doesn't have the problem of trying to make Oracle 19 and 8 speak to each other (through 9,10,11 and 12), but does have the problem of significant change to the language which I had a quick skim over and is similar to the issue we have shifting CF5,6,7,8 and 10 code to the latest version (though we have the extra problem of no one working in the language and a total available contractor base of 3, one of which was ejected from the building last time he worked for us and the other 2 have moved onto only doing CF if Java/.Net work isn't coming their way (which is never)).
And then there's trying to write code for the old versions of CF as well, need help from T'web, fat chance and we lobbed the manuals in an office move years ago.
This of course fuels my skepticism of being able to refactor to modern.

I don't see much point in doing Greenbank's step a beyond refactoring the existing front end to work with a new back end that actually does what's needed and tidily.
But this may also inhibit the design of the new back end (though if it does you've got to be asking if you're actually working on the same business requirements)

Once you have a new back end exposing standard interfaces you can write new front ends that consume it for fun without the legacy restrictions
I make the presumption that unlike work, AUK doesn't have the same pressing need to upgrade away from antiquated platforms. (the gov wanted us off them a few years ago and are now sending round audit teams)

what we all seem to agree on, is AUKs gone for an approach that's the wrong way round, making the shiny front end more important than the business concepts behind it.

[Genosse Brymbo]

I don't see much point in doing Greenbank's step a beyond refactoring the existing front end to work with a new back end that actually does what's needed and tidily.
But this may also inhibit the design of the new back end (though if it does you've got to be asking if you're actually working on the same business requirements)
...
I make the presumption that unlike work, AUK doesn't have the same pressing need to upgrade away from antiquated platforms. (the gov wanted us off them a few years ago and are now sending round audit teams)

If you're refering to step a in Greenbank's discussion of iterative refactoring in #773, then step b gives the answer:

a) Do the grunt work to upgrade php to a modern supported version, which also makes ongoing updates simple
b) This allows you to upgrade mysql and the underlying OS to latest versions, that gets rid of the support/security hangover

That's certainly what you'd do if you were a commercial organisation selling the AUK website as a product.  This goes back to the point Greenbank made in #745.

The problem is that everything it runs on (in terms of software) is old and dated and needs to be overhauled. Software ages, you can't just implement something once and expect it to run forever. Vulnerabilities are found and if you don't keep things up to date then it's a security risk (especially given that it holds data subject to GDPR). The version of the software you're using stops being supported, the upgrade path to the newer versions is not entirely simple, then there's an urgent need to update one item and you have a huge set of dependencies that mean you need to update pretty much everything.

However, as you say, AUK may not see this as such a pressing problem as commercial software vendors do.

[damerell]

Trying to outsource the whole thing is folly, but it's no great surprise that an IT company said they could do it and quoted for it.

Relying on a rag tag band of volunteers with limited time to dedicate to build a system from scratch isnt likely to be particularly successful either.

It's not like AUK's existing Website was... no, wait...

If it took years to get to where it is, you can't go back to nothing and take years over it again.

I'm not sure why you mention that because no-one has suggested doing that.

[Hot Flatus]

Didnt you once post up an unsolicited photo of your arse, or did I dream it?

[FifeingEejit]

I don't see much point in doing Greenbank's step a beyond refactoring the existing front end to work with a new back end that actually does what's needed and tidily.
But this may also inhibit the design of the new back end (though if it does you've got to be asking if you're actually working on the same business requirements)
...
I make the presumption that unlike work, AUK doesn't have the same pressing need to upgrade away from antiquated platforms. (the gov wanted us off them a few years ago and are now sending round audit teams)

If you're refering to step a in Greenbank's discussion of iterative refactoring in #773, then step b gives the answer:

a) Do the grunt work to upgrade php to a modern supported version, which also makes ongoing updates simple
b) This allows you to upgrade mysql and the underlying OS to latest versions, that gets rid of the support/security hangover

That's certainly what you'd do if you were a commercial organisation selling the AUK website as a product.  This goes back to the point Greenbank made in #745.

The problem is that everything it runs on (in terms of software) is old and dated and needs to be overhauled. Software ages, you can't just implement something once and expect it to run forever. Vulnerabilities are found and if you don't keep things up to date then it's a security risk (especially given that it holds data subject to GDPR). The version of the software you're using stops being supported, the upgrade path to the newer versions is not entirely simple, then there's an urgent need to update one item and you have a huge set of dependencies that mean you need to update pretty much everything.

However, as you say, AUK may not see this as such a pressing problem as commercial software vendors do.

If the plan is to dispose of the current web stack in the near term then that's usually considered a "we're working on it" (I've assumed that's a nearish term thing rather than long term)
If the current PHP code couldn't communicate with a more modern version of the DB or through some other interface (e.g. SOAP or REST) then obviously your'e going to have to replace it too, I didn't think it could be that restricted.

[FifeingEejit]

Trying to outsource the whole thing is folly, but it's no great surprise that an IT company said they could do it and quoted for it.

Relying on a rag tag band of volunteers with limited time to dedicate to build a system from scratch isnt likely to be particularly successful either.

It's not like AUK's existing Website was... no, wait...

If it took years to get to where it is, you can't go back to nothing and take years over it again.

I'm not sure why you mention that because no-one has suggested doing that.

[/quote]

It comes from an inference that a rag tag band of developers working part time will be considerably slower than a band of developers dedicated to the cause... obviously they will also be rag tag because developers are but their distractions are likely to be the coffee machine and lunch time rather than life admin and work.

[Greenbank]

It's not years. I'd say that sorting out the existing system and getting it into a better shape so that a third party has a solid base in order to develop a new front end will take considerably less time (even by a rag tag bunch of developers) than the third party implementing the entire back end from scratch and adding a front end onto it at the same time.

The first two phases have not replaced the back end server. The same mysql DB is there for recording results and doing all of the ride processing. Organisers are still using the old site because the organiser functionality has not been implemented on the new site yet. Phase I (the spiffed up front end for calendar viewing and results) shows that they are able to read from the DB to present some data, but I've got no details as to whether this has been done directly hooking into the DB or whether there's some abstraction layer in there.

Phase II concerns taking over the membership data, which is a small slice of the overall pie.

If the plan is to dispose of the current web stack in the near term then that's usually considered a "we're working on it" (I've assumed that's a nearish term thing rather than long term)

The next (and currently last proposed) phase (III) has gone through requirements gathering although the details of this haven't been published yet. Even if phase III involves a complete replacement of the existing aukweb server/site the company that was going to do that work is no more. And until this work is done/tested/accepted/etc the old system has to be kept running.

"AUK has now engaged a lead developer directly." - Another KLAXON goes off.

"Looking forward, this arrangement may or may not be sustainable for the remainder of Phase II and will certainly need reviewed in detail before we commence on Phase III."

So, given the above, I don't see that there can be a valid plan to replace the existing server.

If the current PHP code couldn't communicate with a more modern version of the DB or through some other interface (e.g. SOAP or REST) then obviously your'e going to have to replace it too, I didn't think it could be that restricted.

That's not the case. The current PHP code only runs under a now un-supported version of PHP - that's the problem with that part. The version of PHP can talk to the existing mysql and every new version of mysql with no problems, but upgrading mysql is a problem because of the OS level (I believe), which is restricted by the current version of PHP (IIRC). There's no need to replace php, mysql or anything else, just get it to a state where they can be updated to a recent/supported/vulnerability-free (ha) version.

PHP can't just be updated to the latest version because there are small/dull/boring/search-and-replace changes that would need to be made to the existing PHP code. That's all. Do that work, update PHP, update the OS, update mysql. First bit done, more time bought.

[S2L]

Maybe the underlying problem is that AUK has too many IT experts and therefore there is a belief that something needs to be custom developed, whereas Joe average who doesn't do coding would probably look at some off the shelf affordable solution and sacrifice some of the functionalities, which can probably be sorted by volunteers with excel spreadsheets in their spare time

[FifeingEejit]

It's not years. I'd say that sorting out the existing system and getting it into a better shape so that a third party has a solid base in order to develop a new front end will take considerably less time (even by a rag tag bunch of developers) than the third party implementing the entire back end from scratch and adding a front end onto it at the same time.

The first two phases have not replaced the back end server. The same mysql DB is there for recording results and doing all of the ride processing. Organisers are still using the old site because the organiser functionality has not been implemented on the new site yet. Phase I (the spiffed up front end for calendar viewing and results) shows that they are able to read from the DB to present some data, but I've got no details as to whether this has been done directly hooking into the DB or whether there's some abstraction layer in there.

Phase II concerns taking over the membership data, which is a small slice of the overall pie.

If the plan is to dispose of the current web stack in the near term then that's usually considered a "we're working on it" (I've assumed that's a nearish term thing rather than long term)

The next (and currently last proposed) phase (III) has gone through requirements gathering although the details of this haven't been published yet. Even if phase III involves a complete replacement of the existing aukweb server/site the company that was going to do that work is no more. And until this work is done/tested/accepted/etc the old system has to be kept running.

"AUK has now engaged a lead developer directly." - Another KLAXON goes off.

"Looking forward, this arrangement may or may not be sustainable for the remainder of Phase II and will certainly need reviewed in detail before we commence on Phase III."

So, given the above, I don't see that there can be a valid plan to replace the existing server.

If the current PHP code couldn't communicate with a more modern version of the DB or through some other interface (e.g. SOAP or REST) then obviously your'e going to have to replace it too, I didn't think it could be that restricted.

That's not the case. The current PHP code only runs under a now un-supported version of PHP - that's the problem with that part. The version of PHP can talk to the existing mysql and every new version of mysql with no problems, but upgrading mysql is a problem because of the OS level (I believe), which is restricted by the current version of PHP (IIRC). There's no need to replace php, mysql or anything else, just get it to a state where they can be updated to a recent/supported/vulnerability-free (ha) version.

PHP can't just be updated to the latest version because there are small/dull/boring/search-and-replace changes that would need to be made to the existing PHP code. That's all. Do that work, update PHP, update the OS, update mysql. First bit done, more time bought.

Ah so the upgrade path although a pain isn't too bad.



Sent from my BKL-L09 using Tapatalk

[Kim]

Maybe the underlying problem is that AUK has too many IT experts and therefore there is a belief that something needs to be custom developed, whereas Joe average who doesn't do coding would probably look at some off the shelf affordable solution and sacrifice some of the functionalities, which can probably be sorted by volunteers with excel spreadsheets in their spare time

As a volunteer in the process of sorting out^[1] just such Excel spreadsheets for another cycling organisation in my spare time, I can assure you that that doesn't actually help.  Features will still get added, and the old adage of "you can write Fortran in any language" remains true.  It's just that when your teetering pile of spaghetti is written in Excel macros and passed around between volunteers, it becomes even harder to maintain.


_{[1] I'm fortunate that in this case the task is one of unpicking elements of functionality and handing them over to a properly maintained open-source software suite one at a time.}

[frankly frankie]

... The current PHP code only runs under a now un-supported version of PHP - that's the problem with that part. The version of PHP can talk to the existing mysql and every new version of mysql with no problems, but upgrading mysql is a problem because of the OS level (I believe), which is restricted by the current version of PHP (IIRC). There's no need to replace php, mysql or anything else, just get it to a state where they can be updated to a recent/supported/vulnerability-free (ha) version.

PHP can't just be updated to the latest version because there are small/dull/boring/search-and-replace changes that would need to be made to the existing PHP code. That's all. Do that work, update PHP, update the OS, update mysql. First bit done, more time bought.

Ah so the upgrade path although a pain isn't too bad.

Yeah but as I think Greenbank knows, "search-and-replace" is a bit of an understatement - very little of the code revision required can be done just by a site-wide search-and-replace.  If that was all it took I would have done it long ago. 
When PHP progresses by a decimal point a function (for example, a function to interface and read from the database) gets deprecated and is replaced by another function with a different name which superficially does the same job.  The new function has a different syntax - so a simple search-and-replace is not sufficient.  There are online utilities that seek to address this problem - input legacy code and get back future-proof code - I have experimented with a couple of these with mixed results.  At best any such regenerated code needs to be carefully checked - which really is little better than just rewriting it all from scratch.

For this sort of reason aukweb is stuck at PHP v5.3.29 - a long way short of the current v7.x but even v5.5 would be a huge advance but unattainable without a lot of 'dull' work.  Work that is pointless IMHO unless aukweb has a clear development future.
MySQL we are at v5.5.62 and there are obstacles after v5.6.4 but there may be some merit in updating to that point. After that currently MySQL is at v8 but MariaDb is the way to go, its a highly-compatible and fairly painless (almost) drop-in upgrade from MySQL.  But the db isn't really the main problem.
In terms of server OS - AIUI recent versions of Linux are unable to 'get' versions of PHP as old as v5.3.29 - therefore we also are stuck with an old server OS to match the old PHP version that we need.  The last time we had a complete outage requiring a total re-build from scratch (these things do happen), it was touch and go to find a useful OS version.

[jiberjaber]

Maybe the underlying problem is that AUK has too many IT experts and therefore there is a belief that something needs to be custom developed, whereas Joe average who doesn't do coding would probably look at some off the shelf affordable solution and sacrifice some of the functionalities, which can probably be sorted by volunteers with excel spreadsheets in their spare time

  I agree

[S2L]

Maybe the underlying problem is that AUK has too many IT experts and therefore there is a belief that something needs to be custom developed, whereas Joe average who doesn't do coding would probably look at some off the shelf affordable solution and sacrifice some of the functionalities, which can probably be sorted by volunteers with excel spreadsheets in their spare time

As a volunteer in the process of sorting out^[1] just such Excel spreadsheets for another cycling organisation in my spare time, I can assure you that that doesn't actually help.  Features will still get added, and the old adage of "you can write Fortran in any language" remains true.  It's just that when your teetering pile of spaghetti is written in Excel macros and passed around between volunteers, it becomes even harder to maintain.


_{[1] I'm fortunate that in this case the task is one of unpicking elements of functionality and handing them over to a properly maintained open-source software suite one at a time.}

AUKweb does some operations, like adding up points and automatically detect where there are awards, like SR... but for instance it doesn't do RRTY, for which there is a secretary who does the "spreadsheet" work. I did volunteer for that role, but I was beaten in the queue, meaning these roles are not impossible to fill with a 7K membership.
Whether the points are added automatically or there is a delay due to Kim or Joe having to do them manually, it doesn't really matter, or it doesn't matter enough to justify spending 200 grand.
I can put it in another way... out of 7K members, there are probably around 1K who care about such nonsense like points and awards, the others are there to make up the numbers and have a bit of fun on the bike (or off the bike).
So in other words, we are spending 200 pounds per member to have a system that does a number of operations atumatically (it doesn't yet, actually)... if somebody came to me and asked me for 200 quid to add up my AUK points, I'll probably tell them to get stuffed and so will anybody with a functioning brain.
For a lot less you can get a lot more from the various providers of cycling services online... for instance 200 quid buy you 4 years of Strava premium membership, which does a bit more than simple aritmetics... and even there most people prefer not to spend a fiver a month.

What possessed a bunch of IT bods to think that a parrocchial organisation such as AUK needs a bespoke six figure website is beyond my (and seemingly not only my) comprehension...

[Greenbank]

What possessed a bunch of IT bods to think that a parrocchial organisation such as AUK needs a bespoke six figure website is beyond my (and seemingly not only my) comprehension...

I don't think that's quite right. The majority of the IT bods on here thought it was a bad idea spending £200k on an IT project like this, you can see this if you go back and read the original threads.

I'd phrase it as:-

What possessed the AUK board to think that a parochial organisation such as AUK needs a bespoke six figure website when a bunch of IT bods all screamed no is beyond me (and seemingly not only my) comprehension...

[Whitedown Man]

... the last two posts hit the nail squarely on the head - this isn’t (or isn’t only) an IT issue - it’s a competency and accountability issue, and unfortunately neither is evident.

[S2L]

What possessed a bunch of IT bods to think that a parrocchial organisation such as AUK needs a bespoke six figure website is beyond my (and seemingly not only my) comprehension...

I don't think that's quite right. The majority of the IT bods on here thought it was a bad idea spending £200k on an IT project like this, you can see this if you go back and read the original threads.

I'd phrase it as:-

What possessed the AUK board to think that a parochial organisation such as AUK needs a bespoke six figure website when a bunch of IT bods all screamed no is beyond me (and seemingly not only my) comprehension...

Maybe you are right, I honestly don't know what the board members do for a living... that said, in order to be prepared to spend that kind of money, you need to see that kind of value in a piece of code... Personally, I don't value code, because I can't resell it... that unless it allows for some kind of ROI... I can see why Rapha might want to spend six figure on a website... I can't see why AUK would want to do that

[grams]

AUK's current role is to let people organise and enter events and access results. Without the website none of that happens. I think they're right to identify it as critical to what they do.

The original quote was IIRC for £120,000 all in, for a complete replacement including the pretty new homepage. If this had been even vaguely realistic it wouldn't have been terrible value for money. We were also promised that the new site would have every new bit of functionality people had been asking for, rather than being a bare bones replacement.

The thing that should worry everyone now is that there isn't even the vaguest plan for how many £100,000+ phases are required to get to the point where the plug can be pulled on Aukweb. I think they've only done the easy bits so far, and can see it easily being a 7 digit project.

If the annual surplus is only £40,000 and the long term savings are mostly spent already, how is that ever going to happen?

[Hot Flatus]

What possessed a bunch of IT bods to think that a parrocchial organisation such as AUK needs a bespoke six figure website is beyond my (and seemingly not only my) comprehension...

I don't think that's quite right. The majority of the IT bods on here thought it was a bad idea spending £200k on an IT project like this, you can see this if you go back and read the original threads.

I'd phrase it as:-

What possessed the AUK board to think that a parochial organisation such as AUK needs a bespoke six figure website when a bunch of IT bods all screamed no is beyond me (and seemingly not only my) comprehension...

Well, IIRC, there was the moral panic of AUK holding £250k in reserves, which some felt there was an imperative to be spent.

Then there was the AGM coup orchestrated by VC167 who put in their man, licking their lips at the prospect of spending it, and who not only spent it, but also a whole load of cash that AUK didnt have.

And here we are.

[quixoticgeek]

One of the things that I think might help with looking at this is to stop thinking of it as a website.

What is being produced is a tool to manage the activities that we as an organisation take part in. That could be done as a bestpoke app you down load to your phone, or a spread sheet, or as we have chosen, a website. It is a tool, nothing more nothing less.

In all of this, the actual cost is the thing I have least issue with. I know what these things cost, I know how much I would charge to do something like this, and it's not far from the ballpark of what is being charged (assuming it's a team of about half a dozen people). What I do question is the way it's been managed, and the tools that have been used. I'm seeing the words "lock-in" and "dependency" appearing in meeting minutes in the next few years.

Again, hosting isn't the problem, and that would be easily solved for something within an order of magnitude (either way) of ~£1000/year. That's not what is costing £250k.

Oh agreed, I've not seen a breakdown of how the costs work out, so don't know how much the host ticking over is as a cost of the whole thing. My reason for mentioning it is that my my area of specialisation. It's the area I am most qualified to comment on.

There are several problems with the old/original site, not limited to:-
d) There are various back office interfaces that could do with improvement (membership, organiser, reporting, etc)
e) The front end could do with a revamp and modernisation

I feel that for many people e) is what they see most, and feel is the sole aim of the new site is to make it pretty, not helped by the fact that phase 1 appears to have done only that. I'm a grumpy old fart that hates most of what is considered to be modern web design, so there is very little likelihood I would ever have been positive about the cosmetics of the new site.

What I do object to is when design principle of Verschlimmbessern is used. For pretty much every usecase I have with the AUK website, the new site has made it worse/harder. But then I have yet to sign up to an AUK calendar event.

I appreciate I am not the typical user tho.

The big problem with a) and b), is it makes no visible difference to the end user, That makes it a very very hard sell.

A lot of this thread feels like stopping and asking for directions

"Hi, I'm trying to get to London"
"Oh, I wouldn't start from here" *walks off*

I'm not sure what the solution is.

J