TubeHeartbeat

tubeheartbeat

TubeHeartbeat is a interactive map that I recently built as part of a commission by HERE, using the HERE JavaScript API. It visualises a fascinating dataset that TfL makes available sporadically – the RODS (Rolling Origin Destination Survey) – which reveals the movements of people on the London Underground network in amazing detail.

The data includes, in fifteen-minute intervals throughout a weekday, the volume of tube passengers moving between every adjacent pair of stations on the entire tube network – 762 links across the 11 lines. It also includes numbers entering, exiting and transferring within each of the 268* tube stations, again at a 15 minute interval from 5am in the morning, right through to 2am. It has an origin/destination matrix too, again at fine-grained time intervals. The data is modelled, based on samples of how and where passengers are travelling, during a specimen week in the autumn – a period not affected either by summer holidays or Christmas shopping. The size of the sample, and the careful processing applied, means that we can be confident that the data is an accurate representation of how the system is used. The data is published every few years – as well as the most recent dataset, I have included an older one from 2012, to allow for an easy comparison.

As well as the animation of the data, showing the heartbeat of London as the the lines pulse with passengers squeezing along them, I’ve including graphs for each station and each link. These show all sorts of interesting stats. For example, Leicester Square has a huge evening peak, when the theatre-goers head for home:

leicestersquare

Or Croxley, in suburban north-west London, with a very curious set of peaks, possibly relating to the condensed school day:

croxley

Walthamstow (along with some other east London stations) has two morning rush-hours with a slight lull between them:

walthamstow

Check the later panels in the Story Map, the intro which appears when first viewing TubeHeartbeat, for more examples of local quirks.

This is my first interactive web map produced using the HERE JavaScript API – in the past, I have extensively used the OpenLayers, as well as, a long while back, Google Maps API. The API was quick to pick up, thanks to good examples and documentation, and while it isn’t quite as full-featured as OpenLayers in terms of the cartography, it does include a number of extra features, such as being quickly able to implement direction arrows along lines, and access to a wide variety of HERE map image tiles. I’m using two of these – a subdued gray/green background map for the daytime, and an equivalent darker one for the evening data. You’ll see the map transition between the two in the early evening, when you “play” the animation or scrub the slider forwards.

Additionally, I’ve overlayed a translucent light grey rectangle across the map, which acts to further diffuse the background map and highlight the tube data on top. The “killer” feature of HERE JavaScript API, for me, is that it’s super fast – much faster than OpenLayers for displaying complex vector-based data on a map, on both computer and smartphone. Being part of the HERE infrastructure makes access to the wide range of HERE map tiles, with their distinctive design, easy, and gives the maps a distinctive look. I have previously used HERE mapping for some cities in the Bike Share Map (& another example), initially where the OpenStreetMap base data was low in detail for certain cities, but now for all new cities I “onboard” to the map. The attractive cartography works well at providing context for the bikeshare station data there, and the tube flow data here.

There is some further information about the project on the HERE 360 blog, and I am looking to publish a more deatiled blogpost soon about some of the technical aspects of putting together TubeHeartbeat.

Stats

Number of stations Number of lines Number of line links between stations
268* 11 762

Highest flows of people in 15 minutes, for the four peaks:

Between stations (all are on Central line)
Morning 8208 0845-0900 Bethnal Green to Liverpool Street
Lunchtime 2570 1230-1245 Chancery Lane to Holborn
Afternoon 7166 1745-1800 Bank/Monument to Liverpool Street
Evening 2365 2230-2245 St Paul’s to Bank/Monument
Station entries
Morning 7715 0830-0845 Waterloo
Lunchtime 1798 1130-1145 Victoria
Afternoon 5825 1730-1745 Bank/Monument
Evening 2095 1015-1030 Leicester Square
Station interchanges
Morning 5881 0830-0845 Oxford Circus
Lunchtime 2060 1330-1345 Oxford Circus
Afternoon 5043 1745-1800 Oxford Circus
Evening 1109** 2215-2230 Green Park
Station exits
Morning 6923 0845-0900 Bank/Monument
Lunchtime 2357 1145-1200 Oxford Circus
Afternoon 7013 1745-1800 Waterloo
Evening 1203 1015-1030 Waterloo

* Bank/Monument treated as one station, as are the two Paddington stations.
** Other stations have higher flows at this time but as a decline from previous peak.

I’m hoping to also, as time permits, extend TubeHeartbeat to other cities which make similar datasets available. At the time of writing, I have found no other city urban transport authority that publishes data quite as detailed as London does, but San Francisco’s BART system is publishes origin/destination data on an hourly basis, there is turnstyle entry/exit data from New York’s MET subway, although only at a four-hour granularity, and Washington DC’s metro also publishes a range of usage data. I’ve not found an equivalent dataset elsewhere in Europe, or in Asia, if you know of one please do let me know below.

tubeheartbeat2

The data represented in TubeHeartbeat is Crown copyright & database right, Transport for London 2016. Background mapping imagery is copyright HERE.

Visit the new oobrien.com Shop
High quality lithographic prints of London data, designed by Oliver O'Brien

Putting Cartography Back on the Map – Google Maps Getting Prettier

googlemaps_july2016

There was a time when Google Maps was an ugly ducking. It started life as a road map, and its grey background was decryed at a memorable keynote at the British Cartographic Society annual conference 8 years, contrasting with the classic Ordnance Survey Landranger maps where the spaces between roads were normally full of “something” – be it contours, trees or antiquities. Google’s features, on the other hand, were pretty messy, and often wrong. However, Google has been steadily beautifying its functional map (and correcting it), focusing on the cartography as well as the data, as it turns from a map of roads and POI pins, to a map of everything. 2013 was a big step forward, when the map became vector-based and superimposed features customised to just you. Now in 2016, it’s the look of the map itself that is the focus. Cartography on digital maps is far from dead.

This week, Google has unveiled a the latest update to Google Maps, showing that it is serious about the cartography and colour. The map has a cleaner, more refined look that continues its trend of taking out the detail you don’t need and focusing on the information that you are looking for. The two most obvious changes are (a) a new, brown/orange shading showing “areas of interest” – think high streets and tourist attractions, and (b) smaller roads have had their borders removed and are now simple white lines overlaid on a grey, green or brown background. I have been keen on this technique, using it for OOMap, DataShine and CDRC Maps. MapBox’s basic-style map of OpenStreetMap data also has taken this “white on grey, + data” approach which I am sure has helped inspire Google’s new look. (OpenStreetMap.org has always taken a different approach, with the many contributors wanting their particular mapping visible, it has always looked very busy and colourful. Unlike MapBox and Google Maps, OpenStreetMap.org’s map is to be seen “as is”, rather than acting as a background map upon which colourful project-specific data is intended to be overlaid.)

An accompanying blog post goes into more details about the changes. It includes a nice graphic demonstrating the new colour palette used and how Google are using colour to group and categorise map features, which I’ve reproduced here:

SS3

There is a clear use of complementary colours to balance out the map – the search results and current user interest shown in red, man-made features in pinks and oranges, and natural features in greens and blues, all criss-crossed with the white (and yellow) transport networks. It makes for a map that is logical to look at – and crucially, one that is immediately pleasing to the eye. It doesn’t “shout” at you any more.

One final note – the “Areas of Interest” is a powerful new bit of cartography – it draws the eye to it, and means Google Maps has a significant influence on what parts of an unfamiliar city you are likely to visit. It’s a subtle but key bit of “suggestive” mapping. Bad news for the businesses though that rely on passing trade, and are not in these areas.

Visit the new oobrien.com Shop
High quality lithographic prints of London data, designed by Oliver O'Brien

Population Change in Great Britain 2011-14

popchange_doncaster

The ONS publish small-area population estimates annually, for England and Wales, and the NRS similarly do for Scotland. By taking two of these datasets, we can see how the population of Great Britain is changing – births, deaths, internal and international migration and military deployments/homecomings all act to fluctuate the population.

I’ve taken the 2011 and 2014 “mid-year” population estimates for LSOA and DZs – statistical areas with a typical population of 1000-1500 people – and compared them, to derive small-area population changes. You can see the resulting map here.

In London, a couple of striking patterns appear. Inner West London – Kensington & Chelsea, Fulham, Wandwsorth – is seeing a striking depopulation (orange on the map). This may be due to the tendency of landlords in these wealthy areas to convert old housing stock, that was split into multiple flats, back into houses for the (very) rich. In a few exceptional cases, houses themselves are being knocked together. The unaffordability of the area and its old-age population may also have something to do with it. Further east in Tower Hamlets, increased immigration and a high to-immigrant birth rate may be contribution to the rapid rise in the population here (10%+ in many area – dark purple on the map) in just 3 years. The increase across GB in total, from 2011-14, is 2.1%. Some of the large increases can be due to new university campus accommodation opening up, while large falls are often an indication of housing estates being demolished and redeveloped.

Many cities across Great Britain show a characteristic of newly-desirable city centres increasing in population, as denser housing developments pack people in, while the suburbs decrease in population. The Liverpool/Wirral conurbation is a fine example of this. An exception is Milton Keynes, where no Green Belt constraints its expansions, and new housing estates keep being built in the outer “blocks” of this grid city. Some smaller places with special employment constraints on them seem to be almost universally decreasing, such as Barrow in Furness, as well as Thurso and Greenock, both in Scotland.

Explore the map on CDRC Maps, and Download the data on CDRC Data.

FOSS4GUK Conference

foss4g_atrium

I was at FOSS4G UK 2016 which took place at the new Ordnance Survey buildings in Southampton, a few weeks ago. FOSS4G is short for “Free and Open Source Software for Geospatial”, and the conference focuses on some of the key free GIS software such as QGIS and PostGIS. This was a UK-focused event, following on from the global FOSS4G in Nottingham in 2013, which I was also at. (The next FOSS4G is in Germany in August.)

The OS is a little hard to get to if you aren’t driving there – I ended up cycling right through Southampton from the central station. Once on site though, it’s a lovely new venue, light and airy inside, with the floors and desks of OS cartographers and digital information managers sweeping away to one side of the central atrium, while the conference took place in a couple of large rooms on the other side. Breakout was overlooking the atrium (above). Around 180 people attended, split into two conference streams.

Highlights included:

foss4g_pgrouting
Above: A nice demo of pgRouting usage from Angus Council who’ve switched to open source for asset access mapping. Open and effective code and mapping in a practical, real world context.

foss4g_pgrouting_software
Above: The software used for the Angus Council asset project.

foss4g_chevrons
Above: Add Ordnance Survey Landranger-style hill chevrons to your GIS-created digital maps with this nice bit of code. I love these kinds of talks/demos, which you typically only get at these enthusiast/community-driven meetings like FOSS4G UK. Really interesting bits of code or hacks, demonstrated by the creator who did it just because they thought it would be cool.

foss4g_datashine
Above: I was pleased also to see DataShine getting a mention, specifically its use of OpenLayers UTFGrid for the attribute mouseovers. The talk was by a FOSS/OpenLayers consultant who’s written a book about the mapping platform, which powers most of my web maps. It’s always flattering to get mentions like this, especially as the speaker was probably unaware I was in the audience!

Outside in the atrium there was a mini-exhibition by the talk sponsors, including, intriguingly, ESRI UK, who are presumably keeping an eye on the FOSS4G community, their core business being far from open source (software), even if they have been very keen on demonstrating their products operating on open data.

FOSS4G UK was an interesting and useful couple of days, pulling together the professional and enthusiast geospatial community in the UK to see what’s happening in the space, and a good opportunity to network.

foss4g_mapmakers
Above: “MapMakers”, a housing development next to the old Ordnance Survey office, which is on the way to the new one from the station. The inclusion of the OS grid reference is a nice touch.

Inside HERE

Z

A startup with a billion dollar asset. This is how HERE’s new CEO Edzard Overbeek describes the location services company that is making a striking pitch for being the third major digital mapping and location platform alongside Google and Apple.

HERE has had an interesting recent history. Originally NAVTEQ, one of the major cross-world road network databases, used by various “sat nav” systems, it was bought by Nokia and became Nokia Maps, before being rebranded as Ovi Maps. Nokia then sold its phone business to Microsoft – but as the latter already had Bing Maps, the digital mapping business was spun off into a new unit and sold to a consortium of German car companies. At the time, this perhaps seemed a surprising new set of owners but it has quite quickly become obvious – with self-driving car technology suddenly seemingly closer on the horizon, the need to have a global, highly precise digital map of the world’s streets is suddenly incredibly important – the aforementioned billion dollar asset. Google has been building it up from its initial, low-precision mapping, using its fleet of LIDAR mapping cars, and Apple has been doing the same, arguably starting from an even worse base. HERE has arrived in the space with the highest quality start, having been based on a digital map that is over 20 years old.

The insideHERE Event

HERE was kind enough to invite me to an event, insideHERE, at their European headquarters in the heart of Berlin, for demonstrations of their portfolio, using some of the platforms used recently at MWC, CES and the other major trade exhibitions in the technology and mobile space. They also discussed a few “under the hood” features, and what they are working on right now.

There were three themes, reflecting the three main segments of digital mapping at the moment – business, consumer and auto. A cancelled flight at very short notice (thanks for nothing, Norwegian!) meant that I arrived in Berlin late and so missed the first two. The first can be summarised with the HERE Reality Lens Lens product which provides high quality asset and street furniture mapping for the use and management by local authorities, and the second is encompassed by the HERE mobile app digital app, which occupies the same space as Apple Maps and Google Maps app, aiming to displace these on their respective platforms. This is a challenge of course, as the existing apps are pretty good, so HERE’s unique selling point is that they are designed for offline from the ground up (Google Maps offers this on a slightly more restricted basis, but HERE will be available in offline mode for an area, as soon as you initially load it up online.) Reality Lens and the HERE Offline Maps app are nice pieces of technology that utilise data from HERE’s car data gathering options and make it accessible to public sector and consumer users respectively, but it was clear, both from HERE’s new owners and the comparative length of time used during the day, that HERE Auto is the key sector for the company now.

Geodemographics

HERE have developed geodemographic profiles for car users (drivers/passengers), based on surveys in the USA, South Korea and Germany. Using cluster analysis of the results, they have identified six characteristic types of users, based on how they use cars and other transport options, day to day:

Z-2

Autonomous Navigation Data

Here’s a visualisation of the datasets that HERE use for self-driving cars. These are datasets designed for machines, not people, and the maps of the datasets, shown here, show the breadth and detail of the information used by self-driving cars to determine road information:

9k=

The data in these maps is highly compressed and delivered to cars, anywhere in the world, in cacheable 2km x 2km squares. (N.B. In one of the three pictures showing the maps of these datasets here, there is a mistake with the data shown. Can you see it? It’s obvious – once you’ve spotted it. No, it’s not that the cars on the wrong side of the road, as it’s showing a German autobhan rather than a British highway. Leave a comment if you find it!)

2Q==

Spatial Data Visualisation

HERE also have some nice demo rigs to show their data in a context that is familiar to people, such as using a top-down projection on a 3D model city section, allowing data to be draped over the buildings and street structure:

2Q==-1

9k=-1

Transit Demand Modelling

We also saw a glimpse of a microsimulation-based travel demand model (TDM) for central Berlin, with what-if scenarios possible by placing various objects on the screen visualising the output of the model, such as a rain shower or closed road. The transport mode share will likely continue to adjust in large cities throughout the world, while the street network will often remain static, so such models (and associated visualisations) try and predict what will happen on the ground:

2Q==-2

The other maps shown were in the user interface (i.e. dashboard/HUD) of a car test-rig, which is being used for UX/UI testing of autonomous/mixed-mode driving. I wrote about this in this previous blogpost.

HERE and the Future

Perhaps the most “exclusive” part of the day’s event was an hour long “fireside” chat with the new CEO of the company. As a relatively small group (there were around 10 of us)l, this was an excellent opportunity to grill the top-guy of one of the world’s three from-technology digital mapping providers (as opposed to from-GIS like ESRI or from-paper like the OS). Edzard Overbeek answered every question we threw at him efficiently. I quizzed him on whether indoor digital mapping, the “next frontier” identified by Google at least, will also be a priority for HERE given its new driving focus, to which Mr Overbeek was clear that, in order to be a serious player in the space it needs to be mapping everything, so that a single platform is available cross-use, i.e. if a customer journey ends with a walk through a department store, the platform needs to do the “last 100m” mapping too. It’s clear also that the HERE offline maps app will remain a key part of the company’s offering – not just to realise the value of their existing, long-built-up “consumer-grade” mapping, but to build the “HERE” brand to consumers. Ultimately though, their most important clients are the car companies – both the three that own the company but also others needing a “car mapping operating system”.

Testing Map-Based UIs for Self-Driving Cars: HERE’s Knight Rider

I was kindly invited, earlier this week, to take part in “insideHERE” in Berlin, a small event run at the HERE HQ in Berlin. HERE, being born out of the ashes of Navteq and Nokia Maps, is now owned by a consortium of German car companies. For the special event, HERE’s developers and engineers opened up their research labs and revealed their state-of-the-art mapping and location services work. HERE Auto is making a real play to be the “Sat Nav of the future”, competing directly with Google and Apple to create, manage and augment data between your smartphone and your car. Tomorrow I’ll outline the general visualisation work I saw that demonstrates their high-precision spatial datasets, but first, today, I mention one particular research project which shows how maps will be continue to be a crucial part of driving, even when cars drive themselves.

“Knight Rider” is a test rig, built to simulate a car, where the engineers and UI/UX designers can try out different configurations and locations of controls and maps on a dashboard. They key aspect being tested is how much trust the user can place in the car, based on what they can see and information that is displayed. Testers can sit in the “car” and drive it, to experience map/control designs and, crucially, how it feels to give up the steering wheel but continue to have the confidence that the journey will proceed as planned! Large exterior screens, fans and a windshield provide some depth of realism. The intention is not to create a realistic driving simulator, completely with fully photorealistic buildings and roads, but instead to get the tester as comfortable as possible to evaluate the designs effectively, before they are put in a real test car on the road.

When we saw the rig, it was configured with maps in three places – a short but wide one that wraps across the dashboard, a circular map that sits just to the right of the dashboard, beside the steering wheel, and finally a heads-up display (HUD) that reflects in the windshield, this achieved by a carefully angled screen pointing upwards.

The dashboard map shows a single map, behind the regular digital numbers/dials you would expect on a normal dashboard. The map here switched between a general 3D overview of the journey ahead, when “cruising”, to a more detailed, but still a “helicopter” 3D view, when carrying out manoeuvres such as approaching a destination or a complex junction:

dashmap1

dashmap2

The panel alongside typically shows an overhead map, in a circle with your location on the centre, it rotates as you move:

circlemap
It is also the main drive control panel when not steering, for example if you want to tell the car to overtake a car in front, the AI having decided not to do so already – you are not steering that car here, but “influencing” the AI to indicate that you would like it to do this, if safe:

circlemap2

Finally, the HUD necessarily does not show much information at all, apart from a basic indication of nearby traffic (so that you are reassured the computer can see it!) and any indication of hazards ahead. You mainly want to be looking though the window for the traffic yourself, of course:

hud

The key interaction being tested is changing from human to computer controlled driving, and back. The first is achieved by listening for the comptuer voice prompt, then letting go of the steering wheel once asked to. If you don’t retake control of the car when you need to, for instance as you are changing onto a class of road for which autonomous driving is not available, and you have ignored the voice prompts, then then the car will park up as soon as it’s safe to do so.

It’s an impressive simulator and crucial to shaping the UI of the autonomous cars which are starting to appear on the horizon, in the distance, now.

Photos and video courtesy of HERE Maps.

Mapping Data: Beyond the Choropleth

I recently gave a presentation as part of an NCRM Administrative Data Research Centre England course: Introduction to Data Visualisation. The presentation focused on adapting choropleths to create better “real life” maps of socioeconomic data, showing the examples of CDRC Maps and named. I also presented some work from Neal Hudson, Duncan Smith and Ben Hennig.

Contents:

  • Technology Summary for Web Mapping
  • Choropleth Maps: The Good and the Bad
  • Moving Beyond the Choropleth
  • Example: CDRC Maps
  • Example: named – KDE “heatmap”
  • Case example: Country of Birth Map – concerns of the data scientist & digital cartographer

Here’s my slidedeck:

(or you can view it directly on Slidedeck).

The Great British Bike to Work

Cross-posted from the DataShine blog.

cycle_thumbnail

Here’s a little visualisation created with the DataShine platform. It’s the DataShine Commute map, adapted to show online cycle flows, but all of them at once – so you don’t need to click on a location to see the flow lines. I’ve also added colour to show direction. Flows in both directions will “cancel out” the colour, so you’ll see grey.

London sees a characteristic flow into the centre, while other cities, like Oxford, Cambridge, York and Hull, see flows throughout the city. Other cities are notable for their student flows, typically to campus from the nearby town, such as Lancaster and Norwich. The map doesn’t show intra-zone (i.e. short distance) flows, or ones where there are fewer than 25 cyclists (13 in Scotland as the zone populations are half those in England/Wales) going between each origin/destination zone pair – approximately 0.15% of the combined population.

Visit the Great British Bike to Work Map.

A Map of Country of Birth Across the UK

eastse_countryofbirth

Above: Areas of east and south-east London with more than 8% of inhabitants being originally from (from top to bottom) India (in East Ham), Lithuania (in Beckton) and Nigeria & Nepal (in Abbey Wood).

[Updated] Ever wondered why some branches of Tesco, the ubiquitous supermarket, have an American food section, while others have a Polish food chiller? Alternatively, it might have a catch-all “World Food” aisle, or it might not. The supermarket is, of course, catering to the local community. Immigrants to the UK do not uniformly spread out across the country, but tend to cluster in particular localities.

The latest map that I’ve published on CDRC Maps is a Country of Birth map, which attempts to summarise such communities in one view. It uses the same technique as Top Industry, it maps the most common country of birth (excluding the home nation) of residents in each small area, as of the 2011 Census. The purpose of the map is to identify and map the approximate extent of single-country communities within the UK. For example, to see how big London’s Chinatown is, or whether a Little Italy in the capital still exists.

This map reveals such communities although there is an important caveat when looking at it. I have set out below the rules I applied when constructing it, the most important of which is that only 8% of inhabitants need to share a single country of birth, for it to appear on the map. Bear in mind that, across the UK, 87% of people were born here. These people do not appear on the map, unless they are outside their home nation (and not at all if they are English).

countryofbirth_keyThere are a number of rules I have needed to apply to make this a map that tells an interesting story in a measured and fair way:

  • I don’t map native births – the English-born people in England, Welsh-born in Wales, Northern-Irish born in Northern Ireland or Scottish-born in Scotland. There are almost no areas anywhere in the UK where people born in a single foreign-born country outnumber the native-born. If I did map such native births, then the map would be almost completely dominated by them, and would not tell much of a story.
  • I also don’t map the English-born within the other home-nations, because the population of England is so much larger than in Scotland, Wales etc such that even the small percentage of them moving into the other home nations would dominate the map of Scotland/Wales/NI, if included.
  • I only map a single-country foreign born area if at least 8% of local residents are from that country. This sounds like a low threshold and it is – if an area is coloured a particular colour, it might still have up to 92% of the local residents actually being native-born.
  • The above rule means that some very multi-cultural areas don’t get mapped, because they have a large number of non-native residents, but these are split amongst various countries such that none reaches the 8% threshold.
  • Necessarily, in the source data, some countries are combined together into regions, either for a whole region (e.g. Central America) or for other countries in a region (e.g. Other East Asia, not including China/Japan etc). This is how the underlying Census statistics are represented. This can have the effect of making a result (for a region) appear when it wouldn’t otherwise appear (for any country in the region). However the number of places where this happens is small so it does not overly bias the map.
  • A slight quirk of the census results is that the Scotland and Northern Ireland chose to, based on their own sum populations, aggregate some of the smaller-UK-population countries in a different way. For example, Northern Ireland doesn’t break out “Other Old EU” (e.g. Belgium) and “Other New EU” (e.g. Bulgaria) into separate categories. The Somalian population in Scotland is not presented as a distinct statistic, but it is in NI (and England/Wales). Again, this only affects countries/regions with smaller UK populations so doesn’t overly distort the map.
  • I don’t colour the map where it would be showing data for less than 10 people. This causes a most noticeable rationalisation of the map in Scotland, because the small areas here have a lower population (typically 125 instead of 250 people). This means Scotland’s country-of-birth diversity is a little underrepresented when compared with the other regions of the map.
  • I’ve used colour hues and brightnesses in an ordered way, to group together continents and regions. Greens = UK nations, Olives = Old EU, Browns = New EU, Yellows = North America, Pinks = Central America, Blues = Africa, Purples = Oceania, Reds = Asia. There is no particular meaning to the colours picked beyond this, but be aware that the eye is naturally drawn to some colour hues more than others.
  • If a second country of birth also scores over 8%, but with a smaller local population than the first, then this is shown in striped lines over the first, and labelled as such in the interactive key.

Have a look at the map, and mouse around to find the meaning for the current colour, or see the scrollable key on the right.

Why 8%? I found that dropping this threshold (I tried initially at 5%) results in a lot of “noise” on the map, where only two large families need to move to an area, for it to acquire their birth-country colour. Increasing this threshold (e.g. to 10%, which I tried) results in many of the interesting patterns disappearing.

Interesting, some famous “immigrant” areas of London virtually disappear on this map. Brixton and Hackney are still associated with the Jamaican communities moving there in the 1940s/50s, but, at 8% threshold they virtually disappear. Only at 5% is there a significant community pattern appear. Similarly, Wandsworth and Shepherds Bush are known for their Australian communities but these also almost vanish when moving from 5% to 8%. At a 5% threshold, Hackney and Islington show a “patchwork” effect of integrated multicultural communities of Irish, Turkish, Nigerian and Jamaican-born immigrants. These also disappear largely from the map at 8% threshold. Remnants of the Irish migration to Kentish Town are more obvious.

London remains a fascinating mix where people from many different countries have set up their home in neighbourhoods with established communities and retail that cater for them. While the UK’s other cities have “international” quarters too, none shows the diverse nature of these communities. Virtually every country in the key has a London neighbourhood. (N.B. Places where there are pockets of many nations in a small area in London, and elsewhere, often indicate a student population at a globally well-known university).

Away from London, the Scottish-origin communities in Corby and Blackpool stand out, while the Americans on military bases in East Anglia also dominate the map there. Luton has a Polish, Pakistani and Irish disapora.

As ever, I am mapping small-area statistics, not those for individual houses (I don’t have that information!) and the representation of a particular house on the map is indicative of the local area rather than each house itself. The addition of houses on CDRC Maps maps is intended to make the map more relatable to the population structure of towns and cities, but it can make the data more detailed than it actually is. The map also includes non-residential buildings – there’s no easy way to filter these with the open dataset used, and the great majority of buildings in the UK are residential.

[Update – See this excellent article written by CityLab on this map, which explains some of the above nuances in a better way than I attempted to.]

Below: There is a Little Italy, but it’s in Peterborough now.

peterborough_countryofbirth

London Panopticon

Panopticon Animation

The London Panopticon utilises the traffic camera feed from the TfL API, which recently (announcement here) added ~6-second-long video clips from the traffic cameras on TfL “red route” main roads, to show the current state of traffic near you. The site loads the latest videos from the nearest camera in each compass direction to you. The images are nearly-live – generally they are up-to-date to within 10-15 minutes. If the camera is “in use” (e.g. being panned/zoomed or otherwise operated by an official to temporarily reprogramme the traffic lights, see an incident etc) then it will blank out. The site is basically just JavaScript, when you view it, your browser is loading the videos directly from TfL’s Amazon cloud-based repository.

The Panopticon continuously loops the video clips, and updates with the latest feed from the cameras every two minutes, the same frequency as the underlying source. If you are not in London or not sharing your location, it will default to Trafalgar Square. I’ve added a special “Blackfriars” one which is where the under-construction Cycle Superhighway North/South and East/West routes converge – during rush hour you can already see bursts of cyclists using the new lanes.

Try it at vis.oobrien.com/panopticon and note that it only works on desktop web browsers (I’ve tested it on Chrome, Firefox and Safari). It didn’t work on Internet Explorer “Edge” when I tested it on a PC. It also does not work on Chrome on Android and by extension probably mobile in general. It possibly uses a lot of bandwidth, so this is perhaps just as well.

I’ve named it after the Panopticon, a concept postulated by Jeremy Bentham, co-founder of University College London, where I work, in the 1800s for easy management of prisons. The Panopticon encourages good behavior, because you can’t see the watcher, so you never know if you are being watched. Kind of like the traffic cameras.

The concept evolved from a special “cameras” version (no longer working) of the London Periodic Table, which was itself a follow-on from CityDashboard, both of which I created at CASA. The source is on GitHub.

londonpanopticon

p.s. If you made it this far, you might be interested in a hidden feature, where you can specify a custom location. Just add ?lat=X&lon=Y to the URL, where the X/Y is your desired latitude/longitude respectively, in decimal coordinates. Example: http://vis.oobrien.com/panopticon/?lat=51.5&lon=0.