In an effort to focus my (now rather limited) spare time on a more nuanced set of web projects, I have consolidated some websites.
Three Blogs into One
I have combined two other blogs into this one:
blog.oomap.co.uk – My personal blog, with notes about London, orienteering and cycling. This was my original blog “YepSport” (also known as “Yablog”) and goes right back to 2003.
bikesharp.com – Writing about shared micromobility in the UK. This blog has had less writing than I had planned, as shortly after I started, I was contracted to write similar articles for Zag Daily, for the last couple of years (I have now stepped down).
I still write another blog – Mapping London – which will stay separate for now.
OpenOrienteeringMap Leaves Home
I have also switched OpenOrienteeringMap from my own server/website to David Dixon’s excellent “version 4” rewrite.
OOM has been going since 2009, with saveable maps and a UI redesign in 2013 (version 2), and the ability to generate MapRun-compatible maps (JPGs, JGWs and KMZs) in 2020, capitalising on a boom in popularity for self-run orienteering races during the early months of the COVID crisis and resulting lockdowns (version 3).
However my version had a number of limitations – it used its own copy of OpenStreetMap rather than APIs to the master copy, resulting in delays to people’s edits appearing on the map. It also had very limited contour coverage (10m contours, and GB only). Finally, the actual mapping was inflexible, e.g. you get fences whether you wanted them or not. There was no per-feature toggle.
All of these problems were fixed by David’s rewrite. The only step back is you don’t see the actual orienteering style on the screen, until you generate the PDF. But David has added a Preview button, so the software can go off and generate it on the screen after all – you just need to wait a few seconds.
BikeShareMap and the Meddin Bike-Sharing World Map
Finally, I currently run/contribute towards two websites mapping bikeshare systems around the world. BikeShareMap shows individual docking station statuses, for the ~600 systems for which there is such data. The Meddin Bike-Sharing World Map maps all the bikeshare systems in the world – but just as a single pin, per city.
I have been gradually consolidating the two projects – the first stage has been to use common IDs and split outs for each city, this is now well underway. The second step is to harmonise data input across the two websites, rather than double-keying, as at present. The final stage (which is some way off) will be to integrate the BSM function into the Meddin project.
At the end of each year (normally between Christmas and New Year), there’s a lot to tidyup across Bike Share Map (BSM), the Meddin Bike-Sharing World Map (BSWM) and the UK Shared Micromobility Dashboard. This year, I’m aiming to document all the changes needed, roughly around the time that I make the changes.
Monday:
Superpedestrian (aka LINK) have unexpectedly changed their URL prefix for their GBFS feeds from https://wrangler-mds-production.herokuapp.com/gbfs/{City}/{file}.json to https://mds.linkyour.city/gbfs/[2.2/]{a2}_{city}]/{file}.json. Superpedestrian run the Nottingham escootershare, the fifth largest city for the mode in the UK, so it’s good to have the live data back for that. They also run various other escootershares around the world.
A number of the UK shared e-scooter trials ended at the end of November: Slough (Neuron), Canterbury (Bird), West Bromwich (Voi) and Sunderland (Neuron) – the last due to be replaced by a new Zwings fleet but this will not now launch until the new year – a significant service gap. The Neuron systems had open GBFS feeds (as does Newcastle, their last remaining UK one, although at the time of writing it is returning a 503 server error), so could be tracked live. Canterbury and West Bromwich unfortunately aren’t. I’ve stopped consuming the GBFS feed for Slough and Sunderland.
In addition, there have been very few (if any) escooters visible on many of the Ginger systems, on their app – Milton Keynes (where they compete with Lime and TIER), Hartlepool, Scunthorpe, Whitehaven and even Middlesbrough (the first trial to launch). Only Chester and Great Yarmouth are still going strong. As is often the case with Ginger in particular, there is no news about this. Maybe they are just taking a Christmas/New Year break? The parking locations and zone boundaries still show in their app – but they do for Stafford too, where Ginger pulled out in late 2021. There are GBFS feeds available for some of the Ginger systems, through the Joyride white-label platform they use.
Normally I stop consuming individual GBFS (and other API) feeds during a system’s winter closure period, however the variation with when systems reopen in the new year mean I often miss a few days (or weeks!) when they do restart, so this year I will keep consuming the empty feeds. I am gradually building a guide to indicative months that systems close and reopen, on BSWM, and will correlate this with the data seen in BSM (where available) and publish this additional static data, some time next year.
Lincoln’s tiny Hire Bike website’s certificate has expired, apparently without the operator noticing, so I needed to skip verification of it when getting the website data. To be fair, it did expire on Christmas Day…
Aberdeen’s Bike Issue Bikes ebikeshare finally opened in early November – perhaps with some of their bikes from their recently closed Bristol operation. I have updated the BSWM entry.
Down the coast, the Dundee ebikeshare has gone into a longer winter hibernation this year – more of a temporary closure during a quite time of the year to improve the fleet and operations, rather than snow/ice led, although Scotland is certainly getting some weather at the moment.
I created GPX route files for the challenge. These were created manually in QGIS, using the OpenStreetMap standard “Mapnik” render as a background, by drawing lines, with Google Street View imagery used to check restrictions.
I split each team’s route into 12 stages (so 36 altogether), which were initially each just over 10km and ended at a docking station. Each stage contained between 10 and 40 sequential legs to docking stations. I’m not sure I would trust proper routing engines (based on Google Maps or OpenStreetMap, normally) to have found better routes on each leg between each docking station, than me and Google Street View, largely because many London boroughs have been experimenting a lot recently with Low Traffic Neighbourhoods (LTNs) and modal filters (e.g. two way for bikes/one way for cars). But I did run a TSP solver (RouteXL) on 3 of the stages and in 2 cases it did find a slightly shorter ordering of the legs, within the stage. So I would probably use a TSP solver more for a future iteration of the challenge.
The three route/team files were saved in British National Grid (EPSG27700) GeoJSONs (technically not allowed by the spec) so I could get proper metre distances ($length) auto-updated into a column, for each stage, during planning. The stages had a number column, and were numbered sequentially. Having a number column results in LineStrings in the GeoJSONs and GPX routes/routepoints rather than single MultiLineStrings and GPX tracks/trackpoints. They were then saved as WGS84 GPX files. I (mis-)used a very limited set of column names (name, number, src, desc, cmt), due to the restrictions with the GPX specification – I didn’t want to use GPX extensions.
It was important to have three separate GPX files so that each team would need to load in just a single file to their navigation device and not see docking stations/routes from other teams). But it did make preparations a bit harder for the online map.
The docking stations were imported in via a TSV file, then saved as GPX waypoints (column names again restricted to src, desc, name, and cmt), and the relevant ones were manually appended to the GPX team files. The GeoJSONs were retained as my master editing files, as QGIS cannot easily edit GPX files due to them containing multiple geometry types.
I would certainly like to try a more automated approach to the routing. It did take a substantial amount of time – probably two evenings for each of the three routes, and a further evening for each route to enumerate the docking stations, fine-tune the routes and reorder any sliced up GeoJSON LineString segments (part-stages) back into the correct sequence. The reordering was needed as QGIS would incorrectly reorder parts of the route that crossed over itself, when it was sliced up.
But an automated approach would require a method that deals with docking stations that are just 10m down a no-entry street (so you’d just walk it), which is hard. Currently they are represented as a point defined by TfL through their API (and separately in OpenStreetMap) which may be the location of the “totem pole” kiosk but not the docking points themselves. In routing or GIS systems, the docking station needs to be represented as an area (within which you would walk the bikes) plus a (multi-)line (representing the line of dock points – some of these are quite long – some have significant gaps, and sometimes they are split on either side of a road). Potentially, the point representing a docking station really needs to be an area, and that area can extend up to the nearby road junction to deal with the one-way issue.
Future Improvements
In terms of the general design, a few things could be changed for a future challenge (some of these I mentioned in my previous blog post):
Ensuring that participants are well away from the finish at around the 60-80% stage, so that they are less likely to bail at that difficult time of the day, because the remainder of the challenge is then a kind of “run in” to the finish, rather than routing them away at a late stage.
When participants pass by another docking station twice, they should visit it on the first occasion, not the second time. (An exception is when it is on the wrong side of a dual carriageway, particularly one with a median barrier). Otherwise there is a danger of it being missed on the return.
Build specific meal stops in.
Maximum of 200 docking stations/10 hours per team.
The Web Map
By comparison, building the web map was straightforward, probably just one evening’s work to build the map page itself as a basic OpenLayers map reading in GPX files and with simple browser-based geolocation, and one further evening to build a “team” version of the map that allowed ticking off the stations, the action being stored in a database, and a time string echoed back to the web map (and other viewers, on a Javascript timer) as confirmation. The database had two tables, a summary table with a row per docking station, and an action log which recorded the dock’s TfL ID, timestamp, event type and the submitter’s browser user agent string ($_SERVER[‘HTTP_USER_AGENT’]) in lieu of logins/IDs. It was fairly easy to assign a manually assign each user agent to team, post-event.
Each docking station ended up with 4 identifiers which feels a bit too many, but it kind of made sense:
an integer TfL ID (e.g. 761)
the TfL Name that appears on the totem pole (e.g. Gower Place, Bloomsbury)
a shortcode which was the sequence number and the initials of the first part of the TfL Name (e.g. 37.GP). There were some duplicates across the team. FIN.HS was a special shortcode for the finish for the two teams that didn’t have that as a docking station in their “zone”. One newly added docking station had “A” appended to the sequence number of the previous, rather than having to renumber everything.
a unique sequence code which was the team, stage and docking station order within that stage, (e.g. W02.15). This was used as a logical ordering of the file and to help with assigning each docking station to its stage on the online map.
I also listed an “actual sequence” post-event ordering, e.g. W038, in the final results file.
I could have used the sequence code on the map but felt the shortcode was the most useful concise way of identifying each station to the team as they approached it, and hopefully the simple number would result in any missing out being spotted quickly.
I built a special “diff” webpage that compares our docks file with the live data (via BikeShareMap) every 2 minutes and this alerted us of any new, closed or zero-capacity docking stations, plus a list of full ones. There was one that opened a few days before, but none on the day, thankfully!
Future Improvements
I do think that using fewer intermediate routing points on each leg would be better and would allow for turn-by-turn satnav directions. Having said that, having street names called out is of limited use as they are often hard to spot on the ground, so the breadcrumb trail approach we used worked well.
We had paper maps (just screenshots of the website) as a backup. I never used them, and I think Team South used the website. Team West used them exclusively, with a separate person using the website to tick off.
I would have liked to have had a single source of docking station locations. In the end, they were:
on TfL’s API, which is fed through to a CSV on BikeShareMap every two minutes,
on a CSV file on Github,
as GPX waypoints appended to each team’s GPX routes file, and
in my database for recording times on the ATDMap website.
1 and 2 were automatically compared (see above), 2 could be added to QGIS to compare and generate GPX for 3, and also imported into the database table (4) but this would all be manual steps.
On Monday I spent a lot of time (over 13 hours) cycling between 268 docking stations in London. It was for the All the Docks challenge, as part of Team East, with Joe (Be.EV CCO, and ex-ofo) and Jeyda (fettle CEO). There was also an all-stars Team West and Team South (including Voi, TIER, Zwings, and CoMoUK people – see this blog post for the announcement), and, across the three teams, we aimed to visit all the docking stations in London in a day.
Three challengers: two on Santander Cycles (one docking at every point) and one on their own bike with navigation mount.
Started at 08:51, after getting breakfast nearby.
Two stops for food (at 13:30 and 20:15) plus a shorter stop to buy snacks (at 16:30).
Three lifts (which weren’t big enough for three bikes!)
Finished at 22:21 – 13.5 hours later
GPS says 119km although it added a lot 0f noise around Canary Wharf and a few other places, so I think the actual distance was more like ~113km.
268 docks visited (the 266 assigned to Team East, plus the common finish docking station, plus a short bonus leg to the docking station outside the finish pub).
This works out as visiting a docking station, on average, almost exactly every 3 minutes, for 13.5 hours.
It did take a bit longer than predicted. The target time had been 12 hours (11.5 hours + 30 minutes in breaks) and our average moving speed of 11.9kph was quite close to the 12.5kph in a prediction algorithm I put together. The two hour difference was due to food stops and comfort breaks (75 minutes in total), along with that 0.6kph speed difference (30 minutes extra) and the three legs we I had to jog (15 minutes). Our dock/undock speed got pretty good after Joe (who did 80% of the docks/undocks) got into a rhythm.
My personal bike (which I had planned to ride as a support to the team leader on the bikeshare bike, to do the navigation) had a flat tyre right at the start, which meant I had to hop on a Santander Cycles bike for the entire route. I had a big freak-out when I discovered this (no bike shops in Westfield Stratford City, challenge starting in half an hour) but it was OK in the end – I bought a £20 monthly membership in the app, on the spot, and the ride was more comfortable than I expected, but it did mean I needed to keep fishing my phone in and out of pockets at every docking station.
I also twice forgot to dock at least once an hour, twice, so got two £1.65 overage charges).
A result of my phone (with map) not being mounted on my bike, was I was constantly pocket-tapping the team version of the map as I kept grabbing my phone. This mean I kept ticking off other teams’ docking stations by mistake, particularly Team West’s list. The team version needed a UX tweak so the checkboxes’ labels were not tickable, or have separate tick-pages for each team.
The only significant routing error was at Import Dock in Canary Wharf (no regular Google Street View allowed there, so I couldn’t see in advance) where we ended up at road level, but the docking station was below us, at water level. Taking the bikes down an escalator in the new Crossrail/Elizabeth line station, got us to where we wanted to be.
Proper food stops need to be built in to the schedule, rather than hoping we accrue time and then spend it when we need it. We got hungry in places were there weren’t any quick eateries.
Three legs had to be jogged by some rather than cycled by all, due to infrastructure problems, although we still docked or undocked at each. At the first one, both Joe’s keys stopped operating and the dock was full so I couldn’t dock (with my third account) either. So I cycled to the next docking station, docked, ran back and undocked. A later problem dock wouldn’t release any bikes so I handed mine over and jogged to the next station to get another one. Finally, three from the end, the terminal disconnected after docking successfully, so I ran back to the previous one and went from there to the penultimate one.
One or both of our pair of keys got temporarily blocked several times. With one blocked, using the other, single key resulted in a mandatory 30-second delay before you are allowed to start another journey. Each time, a call to Santander Cycles support fixed the blocked key quickly. Santander Cycles does have a responsive and effective telephone support operation.
One docking station appeared to have been vandalised, with the “slot” on most docks crushed so that the bike couldn’t be docked. Eventually we managed to find one where we could squeeze it in.
Routing people near the finish, then up a hill and away, 10 hours in to the challenge, is demotivating, and coupled with the late time, I think this is what made Team West’s decision for them. A long, clear run into the finish line without big loops away, is preferable! In general, my design for West and East was a long wiggly route up and down, towards the finish, from the outermost parts of the network. By contrast, Team South went near the finish quite early on, before a long tour out to Putney, finally coming back along the river, back into town. I think this latter overall shape of the route probably is better for keeping people going to the end.
A few of awkward wrong-side-of-road docking stations on dual carriageways had to be visited, although generally with traffic lights, we got across OK. The worst link was a short section of The Highway followed by a right turn. This is a horrible road, and the others in Team East took the pavement instead (but were further encumbered by barriers left up from the London Marathon the day before). In retrospect, a walk-bikes route back across a new private development’s plaza would have been better.
A few roadworks made getting to some docking stations tricky. In the end, Team East had four blocked links – resurfacing by the Orbit sculpture in the Queen Elizabeth Olympic Park (where we just cycled through the construction site), park re-modelling in Shoreditch Park (which necessitated using park paths to get around it), Angel Street was closed for building construction (so we went through Postman’s Park instead) and finally St Bride’s Street was completely blocked for resurfacing, but the Poppin’s Court tiny alley diversion was a lot of fun.
The low point was at around 7pm, as the sun set, calculations suggested all three teams were well behind a nominal 9pm finish, and it looked like we would have to bail to make it to the pub. There was also some concern about what it would be like doing this kind of frantic point-to-point riding at night in London traffic, with tired legs, although in the end the traffic levels died down quickly after rush-hour. We also hadn’t had dinner and a proposed food stop at Angel was still 90 minutes away. A conference call between teams was held and a final decision at 9pm would be taken.
We approached the Somers Town Bridge at around its historic 9pm closing time (not sure if it actually does still close) so decided to take a long diversion around.
In the end, just after 9pm, we passed by our team lead’s home (25 from the end) and, with one eye on a very early start the following day, he opted to stop there. Team West had already quit and were arriving at the pub. Team South decided to double down and keep going. Their final stage along the spectacular Embankment at night, with views across the Thames, may have helped with their decision! We carried on for a final section around King’s Cross and, in the end, quite quickly down to the finish at LSE.
What went well
By and large my manual routes worked pretty well. For Team East, there were only a couple of banned turns and one wrong-way street encountered, easily fixable on-the-fly. The teams were pretty happy with my routes, which is good!
Careful planning tried to minimise the number of docking stations on the other side of busy roads, even if this slightly lengthened the route.
I was pleased I could include some of London’s best cycle infrastructure without lengthening the routes. The Olympic Park, Victoria Park, Regent’s Park, Hyde Park, Kensington Gardens, Lower Thames Street and Victoria Embankment (CS3). It was nice, on the day, to unexpectedly discover some of London’s newer protected cycling infrastructure on roads I knew well from pre-protected days.
After a while, we honed our docking/undocking technique and Joe got it down to around 15 seconds. Every 10 seconds saved per dock is 45 minutes saved on the challenge as a whole.
Considering the amount of docking/undocking operations we did, we had very few failed locks/unlocks or disconnected terminals.
My gadget batteries worked out OK. My old Garmin GPS wristwatch lasted 9.5 hours (albeit with no HR or Bluetooth), and then the last bit of route recording was done directly on my phone using Strava in the background. I only used GPS on my phone sparingly, to occasionally locate myself on the web page, but after 11.5 hours my phone’s battery had got down from 95% to 3% so I kept it plugged into a power bank on a cable, for the remainder, and it was fine. Jeyda’s Wahoo’s battery was absolutely fine throughout.
We did visit all 268 docking stations, even if it took us nearly two hours longer than planned.
We all stayed in good spirits as a team, and generally stuck together, although we did split up a bit near the end as different people drove forward to keep the pace going.
I thought we might need an operations person at a desk, checking for changes, handling social media and directing/motivating teams, but actually it worked fine with everyone on a bike.
We crossed the finish feeling fine – it’s the day after that I felt shattered.
Notes for a next time
General consensus in the pub was that having four teams (N, E, S, W) doing around 200 docking stations each, would be more fun and would allow more pub time, food time etc. Team West ended their challenge after around 200 docking stations when it was clear they would struggle to get to the pub before closing time if they continued (and as they had to loop away from near finish at that point). Team South made it just after last orders so Team West bought a round for them in advance. But a decent length social after such an exhausting day is important.
Finishing at the docking station outside the pub would have been better (I had decided against it because the distances across the three teams wouldn’t have matched as well).
Hopping on and off the bike for each dock gets tiring. Particularly as so many of the docking stations are on pavements and facing away from the road, resulting in many kerb hops, jolts and awkward manoeuvres. Possibly alternating the docking between two leads would make this better?
You definitely need one person to have a bike-mounted smartphone or navigation device (Jeyda’s Wahoo device worked pretty well) to do efficient navigation on a mounted stand rather than in/out of pocket.
Having two keys (on two accounts) is essential. As well as cutting down the dock/undock time, it can deal with full docking stations.
Santander Cycles finally launched electric bikes into their fleet, a few days after our challenge. These would be good for support riders, if too expensive for the lead rider (£1 surcharge per leg!)
I/we would definitely invite people to accompany us for a stage or a few legs. We did have Ilma from Fettle along for the City of London legs which was motivating (and she filmed a mini-movie/montage of us)
I’m glad we missed the morning rush-hour and the evening rush-hour while we were in the City was pretty intense too.
Some routing tweaks would be good. See a future blog post here for some technical details on how it was routed and how it could be made better.
Finally, I think the mapping/recording could be automated more. I liked my live-updating map, it was the result of a few evening’s simple Javascript coding, but there’s definitely more that I could do – leaderboards, current location pin, ETAs etc.
One note on helmets – normally I would always wear a helmet on my own bike, but not on bikeshare bikes – we shouldn’t be encouraging it for bikeshare bikes as it significantly reduces their utility and appeal for spontaneous journeys. However – I’m glad I did wear one, because we were on the bikes for 12+ hours, occasionally taking some quite aggressive manoeuvres to get across to the other sides of streets. We had no incidents or even any beeps from drivers, but one person in one of the other teams got bumped by a van. You do feel pretty secure on a Santander Cycle even without a helmet, as they are 25kg, feel incredibly sturdy, and you can never go that fast on them. But for this kind of challenge, hazards were definitely higher than normal and so the extra security of a helmet was welcome.
The Pashley-made bikes (bike numbers starting with 5) make up around 20-30% of the fleet and are definitely better than the older bikes still available, as they are newer. They are slightly lighter, have better lights, a more solid seat adjustment mechanism, and generally just feel nicer. We generally opted for those bikes, and typically stayed on the same bike for hours at a time.
I’ll have one more blog post about the challenge soon, some technical notes about how I put together the routes and some of the issues I faced doing so.
On Monday I will be attempting to visit every Santander Cycles docking station in east London by bike, starting at 9am outside the velodrome in the Lea Valley VeloPark (the “Pringle” from the London 2012 Olympics), cycling over 70 miles and hopefully finishing sometime that evening close to the London Transport Museum in the centre of the city.
It’s the result of an idea by Stephen Bee (of Zwings, an e-scootershare company), Joe Seal-Driver (a mobility expert who formerly ran the ofo bikeshare in London) and Matthew Clark (Chair of CoMoUK, the UK’s bikeshare industry group). They have assembled three teams of three people – the teams will leave east, west and south London at the same time on Monday, each team travelling as a group and docking one Santander Cycles bike at each docking station in the east, south of west part of the network. That’s around 263 docking stations per team – or one every 2-3 minutes, as the plan is, 12 hours later, for everyone to meet on Aldwych.
Why are we doing this? Because it is there to be done, and to see if it is possible to do – perhaps with an eye to a possible single continuous round in the future.
How? Each team will have a lead rider, who has to dock and undock a Santander Cycles bike at every intermediate station, plus two supports on their own bikes (or other hire bikes) – one to navigate and one to tick off the list and update everyone else. TfL was due to launch some pedelecs (electric bikes) into the fleet in September – this got delayed, but if they do appear quietly on Monday, it will be a welcome boost.
The Routes
I’ve created a suggested route for each team – have a look – and the markers should gradually turn green on the day as the teams progress. Some of the teams may be marking progress a different way – but look out for live updates here from at least Team East – subject to battery life and other practicalities. The “official” record of each visit will be compiled by Santander Cycles operations, using their hiring transaction logs, after the event.
Tech considerations – we were hoping to be using routing apps. However – these have proven to be a bit of a challenge. Some have restrictions on the total numbers of waypoints/routepoints, and some insist on rerouting you the long way around… we are aiming to cycle as much of the route as possible, but there are short pavement sections and one-way roads when we’ll need to wheel bikes in one direction to or from certain docking stations. The only long walking section is through Holland Park. So, a more manual approach has been taken. A website with a pre-planned route, and paper backups of the map in case of failing smartphone batteries.
Another issue with routing applications is they aren’t up-to-date with the many one-way/two-way signage from the local authorities. Some areas have introduced Low Traffic Neighbourhoods, and some borough in particular are keen on having one-way roads allow two-ways by bike. Neither OpenStreetMap nor Google Maps is fully up to date with these changes.
There is also the issue of automatic routing between 260+ locations. This is a so-called “Travelling Salesman Problem (TSP)” (the best route between multiple points) which can only be done automatically with heuristics. So, ultimately, the routing has been done manually. Local knowledge, Google Street View and OpenStreetMap have all been useful, plus I’ve visited a couple of links to check their current state. I’ve split each of the three routes into 12 section, each approximately 10km or 1 hour. I did then use an online TSP engine – RouteXL – on some of these sections to see if it could beat my “manual” route planning, and in a couple of places it did spot something clever. But I think my three routes – each around 116km long – will be pretty close to the optimum routes.
The second section “Bow” for Team East.
My preparation is limited – I haven’t even hired one of the bikes for many years – but my commute to work by bike is over an hour each way, which helps. I will also be a doing couple of hours cycling to help at the London Marathon on Sunday, and also (albeit not quite the same type of exercise) I waited 9 hours in The Queue earlier in September. Technical preparation has included buying a Santander Cycles monthly membership, a phone mount, some protein bars and a spare battery pack! Stephen and Joe have also done a test hour. Their rate then suggests it is possible but will be a considerable challenge to have completed our third by the evening. Luck will need to be on our side.
All the Docks should be responsible for 2-3% of all hires on Santander Cycles on Monday.
If all goes to plan, Team East will be heading through Canary Wharf just after noon, through the City of London just before the evening rush-hour, and King’s Cross at around 7:30pm. We haven’t quite decided how/where/when we will be stopping for food – it might have to be on-the-go snacking.
You may be wondering why it’s been so quiet on Bikesharp, the last couple of years… well the reason is that I have been a data journalist on this topic for Zag Daily, an online magazine focusing on shared electric micromobility, particularly in Europe and especially in the UK. So I’ve been writing about the UK e-scooter trials, how bikeshare in the UK is going electric, European market summaries, and various other data-driven aspects of how the industry is evolving and developing, here and around the world.
So far, I’ve had over 70 articles published there, and also supply the live data feed, updated daily and showing the numbers of e-scooters, operators and cities in the UK e-scooter trials – see the numbers panel on the bottom right of the front page. I’ve also published some maps showing the extent of the trials, you can see a variant of one of them here.
It’s just over one year since I last had a detailed look at the worksite for the forthcoming Colne Valley Viaduct, and I was back last week to see what’s changed.
Launch Girder
The viaduct is being built north to south, and this striking red structure, the launch girder, has just been assembled on the earthworks leading to the northern end. It will soon take a slow journey, sliding the couple of miles across the viaduct, atop the newly created piers, building the main bridge deck from the precast sections being created at an onsite factory.
The launch girder sitting on the northern viaduct embankment.Unusual site speed restriction sign.
Piers
These are now appearing regularly, in various stages of completeness stretching along the first part of the viaduct. The plan I believe is to create one a week.
The second pier.The sixth pier, with concrete surface texture detail visible.The seventh pier, being shuttered while the concrete sets from being poured. The ex-Denham Water-ski Club hut is behind.Rebar for the eighth pier in place, nearly ready to be shuttered.
The pair either side of the A412 (Nos. 3 and 4) have a different design, as they are angled, to support the viaduct which will cross the road here at a severe skew angle.
Grand Union Canal
The Grand Union Canal is one crossing that needs to stay open as much as possible – so the haul road will not cross it. Instead, two haul roads will meet either side of it. Here, the photo shows the southern limit of the northern haul road pontoon on Savay Lake, with the forthcoming piers of the viaduct due to appear on the left.
Southern end of the northern haul road, by the Grand Union Canal towpath.
Reflectors for a survey robot have been embedded beside the canal here on its towpath here, at the point that viaduct will cross, to check there’s no movement or subsidence of the canal or towpath here when the piers or viaduct deck go on – particularly important as the land and lake to the west of the towpath is slightly below the canal level.
Survey reflector on the towpath.
Haul Road Pontoon
On the other side of the canal, another haul road pontoon, coming across Broadwater Lake from the south and London. It may advance a little bit more, across the track and cycle route here, and up to the other side of the Grand Union Canal. The viaduct piers will be on the right. It will likely be over a year before these ones get constructed, as this is the southern end of the viaduct. The closed, but not yet relocated, Hillingdon Outdoor Activity Centre, is just to the right of the far end of the pontoon here – spared a direct hit by a late design change in the HS2 route, but still likely to move to be on a different lake.
The haul road crossing Broadwater Lake.Southern haul road pontoon – current end.
I’ve been based at the ESRC Consumer Data Research Centre, a multi-university (UCL/Liverpool/Leeds/Oxford) lab focused on research and provision of specialist UK consumer datasets, since 2015. One of my first outputs was to adapt DataShine, which I’d created in 2013 as part of a previous UCL project, to produce CDRC Maps – to map some of the open datasets we held, and aggregates of some of the more interesting socioeconomic datasets that we produced from the controlled collections.
CDRC Maps is an OpenLayers-based “slippy” (pan/zoomable) map website consisting of pre-rendered raster tiles of choropleth maps of consumer metrics, layered under another raster “context” layer containing roads and labels, and a mask which results in only building blocks being coloured by the underlying choropleth. It served its purpose of showing impactful, pretty and effective maps of our UK socioeconomic datasets, but being a raster based map, with billions of tiles sitting on one of our old servers, it has been showing its age for a while.
The modern web mapping toolstack has moved on, with the rise of powerful web browsers with fast vector rendering, responsive design for smartphones and tablets, and comprehensive GUI frameworks that elevate regular Javascript. CDRC’s requirements have evolved too, with a desire for map visualisation that includes downloadable snapshots, basic analytical functions and filters, rather than the simple view-only concept of CDRC Maps, and a need to embed the map in stories and dataset records, rather than only sitting standalone.
CDRC Maps has also long been hosted directly at UCL, on a local development server. CDRC is a data research centre not a technology centre and there is a desire for use to our server infrastructure for data primarily. The website has long been the most popular public website for CDRC and also is prone to usage spikes due to mass media often finding that maps are a quick way to illustrate a story – or be the story – compared with raw datasets that are less immediately accessible with media deadlines. It was clear that an external host for the sites itself, and ideally the data that powers the site, would be preferable.
To address this and bring CDRC Maps up to date with the new data platform, the centre commissioned Carto and Geolytix to produce CDRC Mapmaker during late 2020. The developers created a Node.js based website that uses the Vue templating framework. Mapbox GL JS 1 is used for the map controls/canvas and the vector tile rendering. The map framework has recently become non-open but there is an open fork, MapLibre, which we will take a look at in due course. The development toolchain has also been brought up to date with industry practice, with proper source code management, continuous integration, rapid development/testing on localhost, and deployment through GitHub.
Map config is in Javascript but this component is separated from the templating Vue/Javascript allowing configuration and setup of new maps to be discrete from the main code itself.
Data is completely separated from the code and there is no server-side processing element for the code. (We do also use an external service, Google Analytics, for our stats). The data is hosted on Carto’s data platform, where a number of datasets are loaded, and also a postcode lookup table. Carto is in fact built on PostgreSQL/PostGIS and provides a management GUI to allow these to be managed independently of the map code.
While the complete (albiet minified) code, config, fonts, images and stylesheets are less than 4MB, the datasets themselves use approximately 7GB of space on the Carto servers. Each geography used (MSOA, LSOA, OA, local authority) has four spatial data files, representing the unmasked choropleth along with three levels of clipping – urban extent (towns/cities), detailed urban (village level) and individual building blocks.
The application is structured around presenting two types of maps – metric maps (which show a various continuous variables associated with a particular dataset, sliced into groups) and classification maps which categorise areas into a single value (sometimes with a hierarchy of levels) and generally include a pen portrait description of the category.
We were delivered six functioning maps and I have gradually worked on extending the codebase and GUI functionality to encompass the wider variety of maps that were on CDRC Maps and that are listed in CDRC Data. Quirks of each additional map have actually meant minor changes to the code in each case to accommodate them, but I am hopeful now that the codebase is broad enough to allow for additional maps to be added in the future with minimal effort.
For this first release of Mapmaker, there are around 30 maps, covering CDRC classifications such as Consumer Vulnerability and the Internet User Classification (IUC), CDRC metric products such as Access to Healthy Assets and Hazards (AHAH) and Residential Mobility (Churn) and some popular government datasets like the Index of Multiple Deprivation (IMD), VOA building ages and Ofcom broadband speeds/availability.
Users can filter maps based on one or more classification categories or on multiple metric value ranges, and a PDF report can be easily produced with a view of the current map, a key and accompanying text and direct link. Clicking many of the maps will not only present the metrics or portrait, but include statistics on proportions in the current administrative area or a custom drawn region. The user interface is deliberately simple with standard pan/zoom controls, map selector, postcode search and layer toggles – that’s it. Planned development in the short term will include an even simpler UI to allow for easily embedding the map in CDRC Data and other CDRC data-led outputs.
CDRC Maps is currently still available for the limited number of maps that show datasets not included on CDRC Data, and it does have the advantage of a pure raster display meaning that some of our controlled datasets which require limited dissemination can be included in this way – on CDRC Mapmaker we would be delivering the dataset to the user’s browser, which is not ideal. Our plan is to de-brand CDRC Maps to provide a home, outside of the core CDRC output, for these legacy maps, in the same way that we have a GitHub repository storing some of our older datasets no longer on our main sites. CDRC is now nearly 8 years old and as the centre’s focus has been refined, not all our older assets have remained central to its mission, but for research reproducibility and historic linking purposes, it is important to preserve these.
We hope CDRC Mapmaker forms a useful visualisation tool for some of CDRC’s many data assets, and its filtering and reporting functionality allow CDRC’s data to be viewed and used in new ways.
High Speed 2 (HS2) will leave London by curving away from the Chiltern Main Line, across the Colne Valley on what will the the UK’s longest railway viaduct. It crosses the Greater London boundary before briefly coming back to ground level and then on into the Chiltern Hills in a long tunnel.
The place between the viaduct and the tunnels is called the Colne Valley Western Slopes. It is here that Align JV, who are building the viaduct in one direction towards London, and the Chiltern tunnels in the other direction towards Birmingham, have created a huge construction site to assist in building both. The site covers several square miles of what were arable fields, and is highly visible from Harefield, a village on the other side of the valley:
The Colne Valley Western Slopes construction site, from Harefield village to the north-east.
The Colne Valley Western Slopes site is very large for several reasons – it houses factories to construct the concrete segments both for the tunnels and the viaduct deck and also the launch podium for the tunnel boring machines (TBMs). It will also be able to comfortably accommodate the chalk spoil coming from the TBMs. Having the production and spoil deposition both on site will substantially cut down on lorryloads bring concrete in and chalk out of the site. A private “junction” has been created on the adjoining M25 motorway, to provide a suitable route for the lorries that do need to arrive.
The upper part of the construction site (focusing on the tunnel itself) viewed from the south-west. The two TBMs (tunnel boring machines) are being reconstructed by the two huge cranes on the left, prior to launching the Chiltern Tunnels boring later this spring. The large factory on the right is the concrete tunnel lining creation plant.
The opportunity is being taken, post construction, to not just restore the arable farmland beforehand, but instead create a large-scale nature habitat – to partially offset HS2’s undeniable impact on the natural environment. HS2 have recently unveiled their detailed plans for the site, which include reconnecting two public paths that have been severed, and plans for new forest to help screen noise and improve biodiversity. A number of set-piece views of the line are also being designed.
A common design style for the tunnel portals, the viaduct, and a new road bridge to accommodate a rerouted road, Tilehouse Lane, that sits in the centre of the site, has been adopted. The design is modern and sophisticated, without distracting from what should return to a natural, rural environment post-build.
HS2 drawing of the Colne Valley Viaduct crossing the Colne River (slightly realigned here) with a planned foot crossing of the river within the Broadwater Lake Nature Reserve here, which will be added. This will be point 5 on the enumerated satellite image below. @HS2.
In the first post in a mini-series, I looked at how High Speed 2 is impacting West Ruislip and Ickenham. In this second piece, I move slightly west, to look at the Colne Valley Viaduct. At 2.2 miles long, this will be one of the biggest structures on the entire line. The viaduct is what carries the line away from the existing Chiltern Line it has been sitting beside, and outside of London altogther, into the Chiltern Hills. As a symbol of leaving/arriving in the capital, and the scenic nature of the valley it crosses, the Colne Valley Viaduct has the potential to be a “tourist attraction” of the railway – both for the people on it, and also users of the natural park that sits underneath it.
The Colne Valley is a very “wet” valley – it was extensively mined for gravel, and as a result there are numerous lakes. The viaduct crosses a number of these, as well as clipping some natural woodland that has gradually built up. A slight planned route change (moving the curve around 100m north) since the initial plans a few years ago, has resulting in less ancient woodland being destroyed, but the line does still pass close to a number of nature reserves, including through one – Broadwater Lake Nature Reserve – that it has unfortunately closed for many years, and outdoor leisure facilities (two of which are in the process of moving).
Harefield No. 2 Lake, the pylons to be moved, and Hillingdon Outdoor Activity Centre, are visible from the National Cycle Route 6 track. The viaduct will pass right across this picture. Points 13-15 in the satellite image below.
It’s not the only railway viaduct in the valley, here’s the Chiltern Main Line viaduct just to the south – incidentally the electricity pylon here will be going soon as part of a power rerouting for the new viaduct, so this particular view will improve soon:
Older viaduct carrying the Chiltern Main Line across the Colne Valley, seen from National Cycle Route 6 beside the Grand Union Canal.
Piling for the piers of the viaduct begins this month and is set to take a number of years, the work moving from west to east, with haul roads currently being built underneath the viaduct. The concrete structures are being built on site. The haul roads will build out from each end, meeting either side of the Grand Union Canal. This break will mean as short as possible a closure for the canal itself – it will only need to close when the deck building girder is directly above it.
The viaduct beside the A412 North Orbital Road, and Denham Waterski Lake (point 3). A strip of trees, forming part of Battlesford Wood, stood here once. There will be a new path and maintenance road here, formed from the haul road underneath the viaduct, which will connect across a new bridge to Broadwater Nature Reserve, opening up more walking routes across the Colne Valley here.
The large, curving and highly visible nature of the viaduct has meant that it has been possible to justify creating a design for it that is not just a simple set of vertical piers. Instead, sweeping arches will carry the line through the woodlands and across the waterways and lakes.
The viaduct will have sound barriers but they will mostly be transparent, except for the section facing a new housing development at Denham Film Studios, so that passengers will be able to enjoy the extensive views across the lakes at ~200mph. At this speed, it will take less than a minute to complete cross the valley on the viaduct. As the viaduct is “bookended” by long tunnels at either end, it will be the most obvious visual clue to south-bound passengers that they are about to arrive in London.
From west to east, the main crossings and affected waterways and buildings are:
Old Shire Lane, a bridleway that also forms the Hertfordshire/Buckinghamshire border. It is currently closed. It will be diverted in a loop to pass under the start of the viaduct. It is due to reopen in towards the end of 2021. A diversion route is further south, via Wyatt’s Covert.
The A412 North Orbital Road. An unpleasant, fast road, which is made even worse at the moment by high dust levels and HS2 construction trucks.
A narrow strip of woodland beside a lake, and the Denham Waterski clubhouse, which is about to be demolished and has a brand new replacement on the opposite shore.
Battlesford Wood, just north of Denham Film Studios.
The River Colne, which will be rerouted slightly here to flow more perpendicularly between two piers of the viaduct. A bridge will also be built here, to connect Battlesford Wood and a new path under the viaduct to the A412 and Wyatt’s Covert, to a nature reserve and Moorhall Road.
Almost the full length of Long Pond (called Long Lake on some maps).
A narrow nature reserve, Broadwater Lake Nature Reserve, which is unfortunately closed for the duration of HS2 construction but will have improved access links afterwards (see above).
Korda Lake – HS2 goes diagonally right across this fishing lake.
Moorhall Road, which goes between Denham Green and South Harefield.
A corner of Savay Lake.
The Grand Union Canal and towpath. A new pier on to Savay Lake will be built here, opening up a view.
A track just east of the canal, that forms National Cycle Route 6.
Diagonally right across Harefield No. 2 lake.
Under a line of pylons that is getting rerouted further east, as it will be too close to the viaduct structure.
Right over the top of Hillingdon Outdoor Activity Centre, which is also closing and moving to an as yet announced location.
The viaduct finally touches down in fields just before Harvil Road.
Here’s where HS2 crossings National Cycle Route 6, between Harefield No. 2 Lake and the Grand Union Canal. Point 12 in the satellite image.
For the last piece in this mini-series, I focus on the huge worksite for both this viaduct and also the Chiltern Tunnels – the Colne Valley Western Slopes.
