Data Graphics Mashups OpenLayers

HE Profiler

This is the fourth in a series detailing the projects I have worked on at UCL in the last academic year.

The HE Profiler is the last of the three “core” school-profiling map mashups that I have developed over the last year – this has been developed over the last few weeks and indeed was finished only today, my final project of the year.

It is designed to be used by university widening-participation administrators, as a graphical tool to discover and evaluate the schools to target for campaigns to encourage university application. To do this, it makes use of two metrics – the OAC demographics of pupils attending each school, and the POLAR score of their postcode – in simple terms a National Statistics demographic describing the likelihood that people from this postcode go to university.

Again it is powered by OpenLayers, displaying point-based vector information on top of Google Maps image tiles, using NPE data for geocoding postcodes. The most interesting thing about this application is I’ve started to explore the very powerful rule/attribute based symbolisation for points available in OpenLayers. This sort of symbolisation will be, I expect, very useful in my next year’s project. I am very impressed with what can be done – some quite GIS-like properties present in a popular and freely available web application.


The graphic above shows target schools for a central-London university, based on the proportion of POLAR1/2 pupils (least likely to go to universities) compared with the rest. Schools with a majority of pupils in this category are coloured red. The area of each circle represents the number of such pupils present. The poor representation at university of the Thames Gateway region can be clearly seen. As an aside, the OAC demographic, not shown here, does not work well for London due to its size – the OAC is calibrated across the whole UK, and it is likely a more specific demographic analysis for London (e.g. LOAC) for schools there, would be more useful.


School Catchments

This is the third in a series detailing the projects I have worked on at UCL in the last academic year.

School Catchments was developed to complement the Education Profiler. Like the Education Profiler, It is powered by OpenLayers, but uses Google Maps tiles as its background rather than custom OpenStreetMap-based ones. For each of the schools in the database, it loads in “catchment maps” for the GCSE and A-Level students. OpenLayers’ vector mapping is used to display them on the Google Map. These reveal the “real” catchment areas for a school, which may or may not correlate with the school’s official catchment area, if available (many schools choose not to publish this information.)

Most schools are roughly in the centre of their catchment area, but geographies, spatial distribution of population – and geodemographics – can all act to distort the contour’s shape. In general, the Roman Catholic and other faith schools have noticeably large catchment areas. Of the regular schools, poorly performing schools often have very small catchments. It should be noted that we only had geodemographic data for the state school sector.

The process of developing the catchment maps was interesting – the known postcodes for each pupil are geocoded and the resulting grid of locations is simplified into a single “contour”, generally surrounding at least 60% of the pupils. One requirement was that only a single contour is produced – problematic when one school serves two small and well spaced villages. The contours were created in R, a statistical programming language popular in the academic community – it’s popular in the department, but quite alien to my Java/PHP/python past. I found the learning curve rather steep and am still a “non-proficient” R coder. Most available documentation for R assumes a level of experience I do not have. Searching for R information in Google is a pain too, thanks to its name.


The generation process is highlighted in the second half of this screen video that my boss produced for a presentation – in the context of the School Profiler that also included the catchments.

The website is also the first eCommerce site I’ve developed (my boss is planning on the catchment vectors being available only through a subscription.) It was interesting getting to grips with the eCommerce paradigm. After struggling for a while to try and make sense of Google Checkout (having assumed, incorrectly in this case, that Google=easy) I switched to using NoChex as the payment processor. It is impressively simple to use, if not well documented, and I got the end-to-end flow set up in only a couple of days.

The site is still in “test” mode and needs some polishing (and the underlying data updated) before a public release.

Mashups OpenLayers

The Education Atlas

This is the second in a series detailing the projects I have worked on at UCL in the last academic year.

This is a mashup, powered by OpenLayers, and using network data from OpenStreetMap (OSM) to provide a “contextual window” on top of choropleths (colour-region maps) representing various educational attributes. Both the choropleths and the OSM maps were created using Mapnik. Data from the NPEMap project is used to provide geocoding (locating from postcodes). Schools from the ShowUsABetterWay competition are available as a simple point-based vector layer.

This project has been through various iterations before ending up as a (sort-of) finished product. An earlier version was briefly demoed at the GISRUK 2009 conference in April at Durham. This was an “all-singing, all-dancing” mashup, which wowed the judges at the conference (it was entered in, and won, the Mashup Challenge competition) with its many layers and features, but was probably too complicated for the intended end use.

The functionality has been split into three different mashups – the first, the choropleths, form the Education Atlas. The school catchment contours are in a separate mashup, School Catchments, which I’ll talk about in a future post. The detailed metrics about each individual school are in a third application.

The choropleths mainly relate to academic attainment and geodemographic background (for GCSE pupils) and A-Level subject choice. Some interesting patterns emerge, for example French is particularly popular in Kent (funny that…) and Geography is more popular in the rural north of England than in the cities – as shown below. The demographic maps show a characteristic pattern of city poverty/underachievement compared with rural areas.


The resulting slimmed-down application is available at, however it is only soft-launched, as the data is quite old, and there are some noticeable gaps in coverage, particularly in Manchester and Hampshire, where state school pupils generally don’t have any sixth-form provision in their secondary schools.

Noteable features, apart from the bespoke black-and-white “network” layer, are the keys, which change depending on the choropleth selected.

I presented some screenshots of the mashup, and talked about how it was made, at the RGS conference in Manchester, in August.

A screenshot of the mashup forms the banner of this blog.

Data Graphics Technical

Spatial Interaction Modelling for Access to Higher Education

This is the first in a series detailing the projects I have worked on at UCL in the last academic year.

My main project through the last year has been to test a hypothesis, developed by Professor AG Wilson, that the flows of students moving from school to university can be approximately by spatial interaction modelling (SIM). Put simply, SIM is a variant of the 300-odd year old Newton’s Law of Universal Gravitation, i.e. the attraction between two masses is related by each of their masses and the distance between them. Replace the masses by the numbers of final-year pupils a school, and a university’s capacity, and make the distance decay exponential instead of inverse-square, and that’s the basics of the model. A similar theory has been applied to great effect by Joel Dearden of CASA, in his retail SIM, which has shown a “tipping point” explaining how supermarkets and out-of-town retail developments have become attractive to shoppers over the last forty years.

Of course, it’s a little more complicated than that, and even with the more complex model I’ve tested, a large number of simplifying assumptions have to be made.

The two main extra parameters that are added to the model are (1) that universities have an “attractiveness factor” above and beyond their size. I have used one of the common university league tables to provide values for this factor. And (2) the distance-decay is not uniform across all types of school students, but varies by their background. By splitting up the final-year school students by demographic, the variation in the distance-decay can be seen, and this is used to calibrate the model.


The seven OAC demographic supergroups are shown here – the horizontal scale is distance and is the same in each graph. (Only English-based school students going to English universities are considered in the study.) The vertical scale is the proportion of students, of that OAC supergroup, in each distance bucket. The actual number of students in each supergroup varies dramatically and this is not shown in the graphs.

The graphs show there is indeed considerable variation between supergroups in the “beta value” of the drop-off if approximated as exponential, and also in the “R-squared” fit to true exponential decay.

  1. Blue collar.
  2. City living – this group strongly favours London, Birmingham and Manchester, i.e. the same or other “big cities” in England, hence characteristic peaks appear at these distances – accentuated by the relatively small school-age population in this group.
  3. Countryside – this group rises before falling, as there is a minimum distance they need to travel to get to even their nearest university.
  4. Prospering suburbs – the lowest beta-value, in other words this group attaches the least importance to school-university distance.
  5. Constained by circumstance – similar to the first group.
  6. Typical traits – the “average” group which encouragingly also has an average looking graph.
  7. Multi-cultural – more distance-sensitive than the others – hence the very steep drop-off. This shows that people living in areas classified as multi-cultural will more strongly desire going to a university that is very local to their home.

Prof Wilson’s theory also factors in the subject that the student is studying (not all universities offer all subjects, and some are most are strong in certain subjects and weak in others), and their attainment at school (i.e. they might really want to study Maths at Oxford, and be at a school very near by, but if they get a D in Maths at A-Level, they aren’t going to be able to do that.)
Universities also come in two types – “recruiting”, where there are more places than students genuinely intending studying there, and “selective”, where there are more prospective students than places. One interesting effect of the recent economic downturn is the massive increase in people applying for university in 2009-10 – UCL saw a 12% increase for undergraduate courses, for example. This has had the effect of making more universities selective.

In order to consider two types in the same model, it was necessary to develop what is known as a “partially constrained” SIM. The details are for a future article, but, put simply, an iterative approach, assigning students to a university and then reassigning the weakest for over-capacity universities, is taken.

I built a GUI in Java – it’s the language I’m most comfortable with for “proper” programming – to quickly visualise the results and compare them with real-life flows. Here’s a bit of it:


This shows the perhaps not very surprising prediction that BIRM7s (multi-cultural school students living in Birmingham) are pretty likely to also go to university in Birmingham (AST = Aston, BCU = Birmingham City University, BIR = University of Birmingham), rather than elsewhere in the country.

When compared with the actual flows:
…the model under-predicts the flow to Birmingham City University, possibly because BCU’s desirability amongst this demographic group is mis-calibrated. Further-education students are also not present in the predicted model, but are included in the actual flows, so the two are not, as presented, normalised.

The model needs to be developed further before it can be presented formally. In particular, attainment is almost certainly a necessary component.


Modelling and Mashing

I’m coming up to the end of my current role at UCL, starting a new role (same department, same lab) on Thursday 1 October. Over the next couple of weeks, I’m going to outline the work I’ve been doing over the last year. The projects I’ll blog about are:

The core project:
1. Spatial interaction model for school to university flow

Core visualisations:
2. Education atlas
3. School catchments
4. HE profiler

Incidental visualisations:
5. Manchester map
6. HEFCE funding map

Preview of my next project:
7. Censusgiv prototype

Other work:
8. A Facebook application for names
9. The Splintdev blogs


M:F Ratio as a measure of a City’s Cycling Friendliness

Okansas links to an interesting study in the Scientific American which relates the cycling friendliness of a city to the male-female ratio of the cyclists in it – the theory being that men are more likely to brave a motor-friendly place while women need more encouragement.

I counted 19 men and 17 women on bikes on my commute into work today, although this was after the normal London commuting time, and a significant part of the commute was not on roads. I suspect that there is more of a male bias on the busier roads and during the rush hour.


Quantum GIS 1.3

A new version of Quantum GIS, the free, open-source and user-friendly GIS, has been released today.

See the official blog for all the details, but the most exciting addition for me is the OpenStreetMap integration. Now, you can download data directly from the OSM servers, into the application. OSM-like stylings are applied to the data to make it look a bit more like a map, and you can easily can view all the tags and relations on each object. You can also edit the data directly in QGIS, as if it was normal GIS data, and then save it straight back to the server. This could potentially make it a good alternative to the Potlatch and JOSM editors that are currently used for the bulk of additions to OpenStreetMap. The integration isn’t perfect – I got a server-side bounding box error on my first attempt out downloading data which should have been caught locally – but it’s pretty impressive nonetheless.

With QGIS’s excellent python integration, it should be possible to write other plugins, to, for example, create well-shaped building outlines with perfect right angles. I think you can do this in JOSM too, but I’ve always found JOSM a rather unfriendly application to use.

Here’s some OpenStreetMap data of my local area, in Quantum GIS, with a road I added highlighted in red:


RGS Annual Conference

As mentioned in my previous post, I was up at the RGS Annual Conference for a day last week. As well as my own session, I stayed to listen to a number of sessions, including the cartography one – titled “Why do Geographers Make Maps?”. This one, a double-session, was popular – the room was packed out, and I enjoyed the talks. But my highlight of the day was an evening trip to the John Rylands Library in central Manchester, for an evening viewing of the Mapping Manchester expedition – which got a lot of publicity on that day, in the national press, because of the “Soviet Invasion of Manchester” maps that form part of the collection.

I was delighted too, to see an old (1980) orienteering map in the collection, and a map showing the locations of all (1000s?) of the pubs in Manchester, was quite eye opening! The building that the collection was in was itself pretty awe-inspiring two – it’s neo-gothic style, and reminds me strongly of the (much older) Duke Humphrey’s Library which is deep in the Bodleian Library complex in Oxford. Basically, it’s straight out of “Harry Potter”.

Conferences Mashups OpenLayers OpenStreetMap

Education Profiling with an Open Source Geostack

I was in Manchester yesterday for the first day of the Royal Geographical Society annual conference. I gave a talk at the session called “New Urban Geography: Evolving Area Classification for Socio-Spatial Generalisation” which was convened by my boss Dr Alex Singleton and chaired by Prof Paul Longley, both also of the Department of Geography here at UCL.

My talk discussed a Web 2.0-style mashup of English school attainment and geodemographic data, which has been put together as an online “atlas” using OpenStreetMap data as a contextual layer, Mapnik to produce the graphics and OpenLayers to display them. The atlas is not yet complete, and the data is a little old, so it’s not being widely promoted yet, but if you are really keen on visiting it yourself you can find the URL by looking carefully in the presentation…

It is here.

[slideshare id=1914330&doc=openlayersandmapnik-090827073034-phpapp01]


OSM gets Isle of Man Government Data

Dan Karran writes that he has received an official, recent (2007) dataset from the Isle of Man government, comprising of a 1:25000 raster with tourist points of interest (POIs), and detailed satellite imagery from 2001, with a licence suitable for tracing into OpenStreetMap. This is great for OpenStreetMap and for the Isle of Man. Although the data doesn’t include individual house shapes, and other fine detail that wouldn’t be on a 1:25000 map, it does allow for complete coverage of the island with a level of detail suitable for routing, POI-finding and landuse mapping.

Now, if the Ordnance Survey could be persuaded to do the same thing for a test area in Great Britain, such as Caithness?