Author Topic: AUK Finances and Website Project was: AUK Chairman Statement  (Read 119345 times)

S2L

Re: AUK Finances and Website Project was: AUK Chairman Statement
« Reply #750 on: 25 February, 2020, 06:48:46 pm »
I'll leave others to describe how this translates to how AUK ended up approaching this whole problem.

I think S2L's question was whether AUK should be in the custom software business at all. Many other national audax organisations (and indeed some AUK organisers) just have a Wordpress site that links to a third party event booking engine, or even just a Paypal link. Whether that approach would be scalable or palatable to AUK is another matter - and academic given the determination to continue with the current approach.


Yes, exactly that... sorry I deleted the post, I feared the usual backlash from the usual suspects, but luckily the message was picked up...

frankly frankie

  • I kid you not
    • Fuchsiaphile
Re: AUK FINANCES AND WEBSITE PROJECT was: AUK CHAIRMAN STATEMENT
« Reply #751 on: 25 February, 2020, 06:53:15 pm »
I would have loved to have helped on the system, and indeed I volunteered myself, but with the proviso that I (along with many other IT experts here) thought that the proposed plan was the wrong thing to do at the time. I would have modernised the existing infrastructure and then looked to improve/enhance in situ. I had an interesting discussion back and forth with the AUK Chair over this and it was decided not to pass on my name to the IT Refresh team due to this fundamental disagreement.

I wish we had gone that way, I really do.  I still wish it but apparently the hole that has been dug is too deep.  And the original project as first mooted (as minuted sometime in early 2013) was very clearly along the lines of "aukweb is rubbish, it doesn't reflect very well on AUK, we need to REPLACE it".
 
Some time later, when I asked the question, all innocent like, of the company that has now gone under - "Will this be a complete rebuild from the ground up then?" the answer was a firm "no, that would cost you twice as much".  This is why we limp along with a hybrid affair across two barely-compatible web servers.
when you're dead you're done, so let the good times roll

S2L

Re: AUK Finances and Website Project was: AUK Chairman Statement
« Reply #752 on: 25 February, 2020, 06:59:33 pm »
It could be that in 2013 aukweb was obsolete, but just like cars, they quickly go from obsolete to classics... I personally find Aukweb quite cool, retro and reassuring... whereas audax.uk looks corporate and dull

FifeingEejit

  • Not Small
Re: AUK Finances and Website Project was: AUK Chairman Statement
« Reply #753 on: 25 February, 2020, 07:25:48 pm »
It could be that in 2013 aukweb was obsolete, but just like cars, they quickly go from obsolete to classics... I personally find Aukweb quite cool, retro and reassuring... whereas audax.uk looks corporate and dull

Good luck getting the carbs tuned

quixoticgeek

  • Mostly Harmless
Re: AUK Finances and Website Project was: AUK Chairman Statement
« Reply #754 on: 25 February, 2020, 07:48:56 pm »
It could be that in 2013 aukweb was obsolete, but just like cars, they quickly go from obsolete to classics... I personally find Aukweb quite cool, retro and reassuring... whereas audax.uk looks corporate and dull

"When people say legacy, they mean stuff that works"

J
--
Beer, bikes, and backpacking
http://b.42q.eu/

Davef

Re: AUK Finances and Website Project was: AUK Chairman Statement
« Reply #755 on: 25 February, 2020, 08:00:27 pm »
It could be that in 2013 aukweb was obsolete, but just like cars, they quickly go from obsolete to classics... I personally find Aukweb quite cool, retro and reassuring... whereas audax.uk looks corporate and dull

"When people say legacy, they mean stuff that works"

J
“We will start again, a clean sheet, we won’t make the same mistakes as before” and behold it came to pass, they discovered many new and innovative mistakes.

I like the old site (despite the fact I am lurking in the background of one of the photos in the new one). If the issue was of obsolete technology then moving the old site onto new technology might have been the way to go. Perhaps the new site is more aesthetically pleasing to those that appreciate these things in the same way my bike looks prettier without its mudguards.


Sent from my iPad using Tapatalk

quixoticgeek

  • Mostly Harmless
Re: AUK Finances and Website Project was: AUK Chairman Statement
« Reply #756 on: 25 February, 2020, 08:02:12 pm »
I was one of those questioning the costing of all of this new site stuff. I'd like to be in a position to volunteer to sort it all out, but there are only 168 hours in a week, and for some reason I can't fathom the places I buy food from need paying.

I'd offer to provide hosting if I thought it would be useful, but I can only offer Linux based offerings, and I don't know how that would work with it being in Amsterdam given recent events.

But, above all else. If it broke while I'm in the middle of a 2 week race. It wouldn't get fixed until I got back, which for a site with thousands of users, that is rather sub optimal.

We're with the classic conundrum of "cheap, on-time, good" choose no more than 2.

My impression of the supplier we've chosen is pretty poor, and I'd question the mechanism by which they were chosen, but I do not question the idea of paying someone to do this sort of thing. Provided the contracts are negotiated properly, due diligence is done, and SLA, SLI, and SLO are all chosen correctly.

But ultimately with all projects like this, the customer doesn't really know what they want, and what they need is not what they think they want. Thus they get something that is unfit for purpose on many levels, usually delivered late, and always over budget.

Quick note re "take wordpress and do X" solution some suggested. That's kinda the formula here. A .net based CMS has been taken, and then customised to work for us. Unfortunately by doing so we are now locked into that CMS, which is a problem, as it doesn't appear to be one where experience in it is all that common.

Web development as an industry is a horror show, and it's not likely to improve any time soon.

J
--
Beer, bikes, and backpacking
http://b.42q.eu/

FifeingEejit

  • Not Small
Re: AUK Finances and Website Project was: AUK Chairman Statement
« Reply #757 on: 25 February, 2020, 08:48:50 pm »
"When people say legacy, they mean stuff that works"

The problem with Legacy that currently works is it can very suddenly not work and the only result from support (whether that is commercial or Stackoverflow) is a load of laughing.
Like when the Oracle 8 and Coldfusion 5 stack systems at work started creaking a few years back...
That's not to mention the incredibly badly written .NET 1.1 systems that I support through black magic and a windows XP VM that officially doesn't exist.

Martin

Re: AUK Finances and Website Project was: AUK Chairman Statement
« Reply #758 on: 25 February, 2020, 08:51:24 pm »
 :thumbsup: to having most of the user portal on the new website soon but if the bricks and mortar of the "old" database (which I use exclusively as a rider and organiser; it's not as others have said just there to look up obscure records it's the entire online history also going forwards of all AUK rides) is still the same system I can only ask why did it / will it need replacing?

if it can be kept going with a bit of volunteer tinkering and the occasional shilling in the meter that works for me  :)

Phil W

Re: AUK Finances and Website Project was: AUK Chairman Statement
« Reply #759 on: 25 February, 2020, 09:24:05 pm »
Quick note re "take wordpress and do X" solution some suggested. That's kinda the formula here. A .net based CMS has been taken, and then customised to work for us. Unfortunately by doing so we are now locked into that CMS, which is a problem, as it doesn't appear to be one where experience in it is all that common.

Web development as an industry is a horror show, and it's not likely to improve any time soon.

J

Depends on what you mean by customisation.  With Wordpress you do not customise it by changing the core Wordpress code what so ever. You use themes to style it, and write plugins to enhance or introduce new functionality.  So in effect you can keep updating the core code on your website ad Infinitum as it is updated by the Wordpress core team at Automattic.

If the Umbraco code has been customised so you can’t just update the core as it gets updated then that’s a bad place to be indeed. You end up with a core code base that very rapidly ends up in an evolutionary dead end and out of support. Sure that can’t be the case.

As you’ve pointed out Umbraco has a much smaller set of people with skills in it, than Wordpress. Wordpress powers about 25% of the World’s websites or something like that.

We are where we are though with the support arrangements now? We are kind of caught between volunteer support and professional support.


FifeingEejit

  • Not Small
Re: AUK Finances and Website Project was: AUK Chairman Statement
« Reply #760 on: 25 February, 2020, 10:26:58 pm »
How many wordpress installations are actual commercial CMS systems rather than the original blogging system?
And how many have significant amounts of php work added to them for corporate needs?

At least umbraco was designed for the job of being a extensible CMS. The. Net and IIS platform it runs on and is extended in is hardly niche either.
You'd assume its got a decent sdk or api too... So in theory anyone that's done c or basic style coding can write stuff for it.

At least it's not entirely written in a bunch of "latest big thing" languages and platforms that ultimately die the following year.



Sent from my BKL-L09 using Tapatalk


Phil W

Re: AUK Finances and Website Project was: AUK Chairman Statement
« Reply #761 on: 25 February, 2020, 10:59:57 pm »
How many wordpress installations are actual commercial CMS systems rather than the original blogging system?
And how many have significant amounts of php work added to them for corporate needs?

At least umbraco was designed for the job of being a extensible CMS. The. Net and IIS platform it runs on and is extended in is hardly niche either.
You'd assume its got a decent sdk or api too... So in theory anyone that's done c or basic style coding can write stuff for it.

At least it's not entirely written in a bunch of "latest big thing" languages and platforms that ultimately die the following year.



Sent from my BKL-L09 using Tapatalk

They are both designed to be extensible. I was really commenting on the fact you really don’t want to be customising, whatever the base COTS. We will never know how many are commercial with either platform as both are open source and can do it yourself or offered as various hosted options etc.

Anyway these choices are mute, Umbraco it is with the ex lead developer of the company that went bust.

Re: AUK Finances and Website Project was: AUK Chairman Statement
« Reply #762 on: 26 February, 2020, 12:01:30 am »
...
I'd offer to provide hosting if I thought it would be useful, but I can only offer Linux based offerings, and I don't know how that would work with it being in Amsterdam given recent events.

But, above all else. If it broke while I'm in the middle of a 2 week race. It wouldn't get fixed until I got back, which for a site with thousands of users, that is rather sub optimal.

We're with the classic conundrum of "cheap, on-time, good" choose no more than 2.
...

Again, hosting isn't the problem, and that would be easily solved for something within an order of magnitude (either way) of ~£1000/year. That's not what is costing £250k.

There are several problems with the old/original site, not limited to:-

a) The existing software stack is outdated[1] and the inter-dependencies such that getting everything updated is a sizeable task in of itself
b) The existing data store schema could do with a redesign (as many "new" items like ECEs have been worked into the existing system using various tricks) and the existing data migrated to the new schemas
c) Given the implementation there's little/no (or at least not enough) abstraction between the front end code and the DB schema
d) There are various back office interfaces that could do with improvement (membership, organiser, reporting, etc)
e) The front end could do with a revamp and modernisation

(b) and (c) are the core of the hidden problem and make fixing (a) a bit harder. Once you get these sorted (by implementing a proper abstraction between the DB and the front end, which can be done in situ and by volunteers) then (d) and (e) become considerably easier and are much easier to outsource.

The outsourcing may end up replacing the whole backend but at least it'll do so from a clearer and more understandable base. Trying to outsource the whole thing is folly, but it's no great surprise that an IT company said they could do it and quoted for it.

But, fundamentally, if there's no proper abstraction and your front end (i.e. php) code is responsible for both accessing the DB directly and serving the HTML then you're in a whole world of pain and misery. Any small change becomes a huge task searching for other places where a schema change could break something else, so schema modifications are avoided in preference to schema additions and overloading and it just snowballs.

Given the above I would have done them in the order of (c)+(a) [but primarily (c)] internally, and then (d)+(e) could be outsourced and done alongside (b) if the first bit (c+a) was done properly.

From a end user perspective then (e) is the key part.
From a back end user (i.e. membership secretary, recorder, organiser) then (d) is quite important.
From a maintainability point of view (a), (b) and (c) are really important.
But from a security/GDPR/future point of view (a) is the absolute priority.

1. In terms of versions, nothing specifically wrong with the individual packages/technologies chosen but they're just old and updating one would mean lots of tweaks leading to bigger fixes and possibly even bigger rewrites of various chunks.
"Yes please" said Squirrel "biscuits are our favourite things."

Re: AUK FINANCES AND WEBSITE PROJECT was: AUK CHAIRMAN STATEMENT
« Reply #763 on: 26 February, 2020, 12:55:19 am »
Im sure they will all get their shoulder to the wheel soon enough, rather than expending efforts on petty and ill informed comment on this forum.
It doesn't help to try and put rivets into a bridge that will soon fall down.
Quote
With so many people possessing all the correct skills required for exactly this situation
This is a regular straw man in this debate. If you are proposing to sue someone over a matter of land ownership, and your friend the criminal lawyer spits the coffee out of their mouth when you tell them about it, the right answer is not to ask them to take over the case since obviously they know how to win it.
I wish we had gone that way, I really do.
That was obviously a better option.

I didn't renew my membership; I don't mind AUK using my money for audax (even after, as was the case at the time, 3 years out of work) but I do mind them just feeding it to consultants. However, I did hope they'd at least get a Pyrrhic victory out of this.

FifeingEejit

  • Not Small
Re: AUK Finances and Website Project was: AUK Chairman Statement
« Reply #764 on: 26 February, 2020, 03:33:14 am »
Trying to outsource the whole thing is folly, but it's no great surprise that an IT company said they could do it and quoted for it.

Relying on a rag tag band of volunteers with limited time to dedicate to build a system from scratch isnt likely to be particularly successful either. If you want to deliver you need 1 dev day to be 1 day, not 3 months of tinkering.

Trying to mix or swap development teams midway through a project isn't going to look pretty either regardless of whether it's 2 full time teams or what.
Argumentes over responsibility are inevitable and when the other company claims credit for what you rewrote because they got it "wrong"... (been there, watched 2 devs go apoplectic and quit...)

I'm not sure how the actual software build could be done otherwise.
Initial Content creation could be done in house, but it's a lot more work than day to day content creation, and the people doing it have other auk stuff to do along side life.

Would be interested to know how others would go about spliting it between outsourced and other sourced.


As for whether an insitu redevelopment was realistic, I don't have the info from the discussions or the tech details.
But theres only so much you can do on the same stack, I've heard "PHP has totally changed give it another shot" but it sounds like that means a new website with little portability of existing code, and if there's no separation of concerns or seams in the existing code (which given the code bases age I expect is the case) in the code you're starting from scratch anyway.

And if the data concepts are messy as we keep being told they are, we'll a new dB schema is almost a certainty... And oracle have all but killed MySql. (im assuming lamp here), postgres or Maria? Why did postgres copy oracle's sql/pl-sql mess?

Oh and the A appears to be out dated too, nGinX seems to be the rage

Did I mention at work we're currently trying to migrate loads of ancient apps to a stack that is a) in support and b) not laughed at by the security inspectors?
It's going well enough that rewriting systems is now on the cards...

Sent from my BKL-L09 using Tapatalk

Re: AUK Finances and Website Project was: AUK Chairman Statement
« Reply #765 on: 26 February, 2020, 03:40:49 am »
Trying to outsource the whole thing is folly, but it's no great surprise that an IT company said they could do it and quoted for it.
Relying on a rag tag band of volunteers with limited time to dedicate to build a system from scratch isnt likely to be particularly successful either.
It's not like AUK's existing Website was... no, wait...

FifeingEejit

  • Not Small
Re: AUK Finances and Website Project was: AUK Chairman Statement
« Reply #766 on: 26 February, 2020, 03:55:41 am »
Trying to outsource the whole thing is folly, but it's no great surprise that an IT company said they could do it and quoted for it.
Relying on a rag tag band of volunteers with limited time to dedicate to build a system from scratch isnt likely to be particularly successful either.
It's not like AUK's existing Website was... no, wait...
If it took years to get to where it is, you can't go back to nothing and take years over it again. You need all the bits you want in the system ported over and available from day 1.


Sent from my BKL-L09 using Tapatalk


S2L

Re: AUK Finances and Website Project was: AUK Chairman Statement
« Reply #767 on: 26 February, 2020, 08:21:24 am »
It would be interesting to find out how much CTT (cycling time trials) spent for their website, which also does unusual things

Davef

Re: AUK Finances and Website Project was: AUK Chairman Statement
« Reply #768 on: 26 February, 2020, 08:57:33 am »
Maybe with the demise of the existing developers it would be a good time to stop and take stock.


Sent from my iPad using Tapatalk

Re: AUK Finances and Website Project was: AUK Chairman Statement
« Reply #769 on: 26 February, 2020, 09:04:48 am »
Having been on projects that delivered functionally similar app/web/data linkage things for £40k, £100k, £180k and £1,000,000 I suspect that the only people who know how to cost this stuff are people who know what it should involve. I get the sense that there a few of them in this thread, and that they haven't been listened to.

frankly frankie

  • I kid you not
    • Fuchsiaphile
Re: AUK Finances and Website Project was: AUK Chairman Statement
« Reply #770 on: 26 February, 2020, 09:13:17 am »
There are several problems with the old/original site, not limited to:-
...
From a end user perspective then (e) is the key part.
From a back end user (i.e. membership secretary, recorder, organiser) then (d) is quite important.
From a maintainability point of view (a), (b) and (c) are really important.
But from a security/GDPR/future point of view (a) is the absolute priority.

I'm very impressed with Greenbank's grasp of the issues and his clear analysis.  Everything he says in reply #762 is spot on, from a technical point of view.

But of course there is also a political layer to all this.  Whatever was the motivation for the initial project (in 2013) seems to me to have been largely lost - none of the people who were instrumental in getting it started, are still actively involved in the audax.uk project.  In fact from my perspective there's been two or three generations of churn in personnel on the AUK side, and more than that on the contractors' side.  I don't think anyone really knows why we're doing this any more.

It would be interesting to find out how much CTT (cycling time trials) spent for their website, which also does unusual things

Of course we can learn from other similar operations.  Though the aukweb site itself has been very influential in that regard, certainly the fundamental core data model has been much emulated by other similar organisations globally.

Maybe with the demise of the existing developers it would be a good time to stop and take stock.

They went down on 2nd Jan - several weeks ago now.   I would hope something of that sort happened then - but I don't know for sure, I'm not involved at all with the new project, other than adapting aukweb to fit in, facilitating controlled data access, etc. 
While we're throwing analogies around, my position is like servicing and repairing a loved old car, just to get it through an MoT so that it can be driven to the scrapyard.  It's not a good place to be.
when you're dead you're done, so let the good times roll

megajoulesexpenditure

  • More Pedalling Less Paperwork
Re: AUK Finances and Website Project was: AUK Chairman Statement
« Reply #771 on: 26 February, 2020, 09:19:02 am »
If it wasn't so very worrying for the continued existence of AUK it could be funny ::-) :-[

If you substitute website for computer does it not have shades of 'Deep Thought' if anyone still remembers Hitchhikers Guide to the galaxy?

Re: AUK Finances and Website Project was: AUK Chairman Statement
« Reply #772 on: 26 February, 2020, 09:57:50 am »
As for whether an insitu redevelopment was realistic, I don't have the info from the discussions or the tech details.
But theres only so much you can do on the same stack, I've heard "PHP has totally changed give it another shot" but it sounds like that means a new website with little portability of existing code, and if there's no separation of concerns or seams in the existing code (which given the code bases age I expect is the case) in the code you're starting from scratch anyway.

My working assumption for in situ redevelopment is that the code that exists *works* and as much of it as possible should be left alone. The things I've heard to be wrong with it could be solved with a few days of strategic search and replace.

It would be nice to have a new tidied schema, but not if it ends up costing you a million quid, which is where this project is heading.

(and AFAIK all new code developed so far is built on the old schema, so all that code is *already* legacy)

Re: AUK Finances and Website Project was: AUK Chairman Statement
« Reply #773 on: 26 February, 2020, 10:04:48 am »
As for whether an insitu redevelopment was realistic, I don't have the info from the discussions or the tech details.
But theres only so much you can do on the same stack, I've heard "PHP has totally changed give it another shot" but it sounds like that means a new website with little portability of existing code, and if there's no separation of concerns or seams in the existing code (which given the code bases age I expect is the case) in the code you're starting from scratch anyway.

And if the data concepts are messy as we keep being told they are, we'll a new dB schema is almost a certainty... And oracle have all but killed MySql. (im assuming lamp here), postgres or Maria? Why did postgres copy oracle's sql/pl-sql mess?

Oh and the A appears to be out dated too, nGinX seems to be the rage

That's pretty much it, a simplification (as I understand it) would be something like:-

1) The DB schema can't be unmessied easily as there's no abstraction between it and the front end code
2) The version of php is old and can't be upgraded without reasonable grunt work (nothing hugely technical, just endless stuff like https://www.php.net/manual/en/migration70.incompatible.php)
3) There's nothing inherently bad about mysql, changing it for the sake of changing it isn't required and the support path looks good regardless of what Oracle do, mysql isn't going to suddenly disappear
4) Apache/nginx is just a detail. Apache is perfectly serviceable in this regard, we're not worried about serving 10k requests/second here.
5) The OS version is ancient, but can't be upgraded because of the version of php (or something else)

So, how could you fix it?

I disagree that you need to start from scratch, indeed it's far better in situations like this to go for iterative refactoring and abstraction than for a big bang replacement project.

a) Do the grunt work to upgrade php to a modern supported version, which also makes ongoing updates simple
b) This allows you to upgrade mysql and the underlying OS to latest versions, that gets rid of the support/security hangover
c) Design a new DB schema and use this to put in a decent abstraction layer between the front end code and the existing DB. The front end code shouldn't have any references to specific DB columns, ideally do something like making the abstraction layer return JSON and make the front end code parse this JSON to present it to the user - implementing the abstraction against the existing DB will generally lead you to see where the obvious flaws in the schema are
d) This allows you to make easier modifications to the DB schema to move towards your ideal goal, as long as the JSON returned stays in the same format - this is your abstraction
e) This JSON interface can now be handed to a third party developer who can use whatever JS toolkit they want to make the end user website look pretty

Tasks (a) to (d) lend themselves perfectly to a team of volunteers who can slowly chip away at it piece by piece, and there are a whole bunch on here who would have helped out on it had they not gone with handing the whole thing to a third party who started, pretty much, with the front end first and promises to deal with the rest eventually.

More importantly tasks (a) and (b) can be done in isolation (with an anonymised/dummy DB) which makes testing much easier. When there's confidence that the changes are good then this can be swapped in. Irrespective of the grand plans by AUK this work (a+b) needed/needs to be done anyway.

To the point of "well, experts here should step in" - many tried. As I said I had a conversation with the AUK Chair about it in 2018 and offered to help out on the equivalent of a-d above in order to prioritise the parts that really matter and then make (e) much easier, and to move that decision point further on down the line, but the board made a different decision.

Anyway, we'll see what lands in April and what decision the board make between now and then. As others have said I hope they take this as an opportunity to reassess the position. The "cash positive" bit of the comment really pissed me off, as if it is in anyway good that they might only be spending £50 on a £10 meal instead of £100.

I'll continue my AUK membership even though it makes little sense for me (I don't ride many if any Audaxes at the moment, I don't really care about my results being online, I don't really read Arrivee and I'd much rather receive it electronically anyway, and however much I think I may I'm unlikely to ride LEL or PBP again in the near future.) I don't think AUK is at risk at all, as I said before, organisers will still put on rides and riders will still want to ride them - politics about a website doesn't really change any of that.
"Yes please" said Squirrel "biscuits are our favourite things."

Re: AUK Finances and Website Project was: AUK Chairman Statement
« Reply #774 on: 26 February, 2020, 10:09:27 am »
If it wasn't so very worrying for the continued existence of AUK it could be funny ::-) :-[

If you substitute website for computer does it not have shades of 'Deep Thought' if anyone still remembers Hitchhikers Guide to the galaxy?

All AUK actually needs is a correspondant accredited to ACP and a few volunteers to run events.  Other stuff is nice but not essential.