Smart cities and the data deluge

Abundance of data about individuals and their environment

"Big data revolution" (Mayer-Schonberger and Cukier, 2013)

"Data deluge" (Kitchin, 2013a)

Smart cities

cities that "are increasingly composed of and monitored by pervasive and ubiquitous computing" (Kitchin, 2013a)

This talk presents some LIDA projects that aim to make use of these data

Modelling Footfall with Machine Learning

Aim: to better understand footfall in Leeds city centre, using computer models and footfall camera data

Methods

Data analysis & visualisation

Machine learning

Outcomes

Analysis of changes in footfall patterns over time

A model of footfall, able to quantify the success of events

Indications of most important drivers

Modelling Footfall with ML

Footfall Data

8 cameras installed between 2007 and 2009

Track movement of people through their field of vision

Provide counts of number of passers-by per hour

Cover a relatively small area of the city centre, not good for wider dynamics.

Modelling Footfall with ML

Side note: Temporal Change

Saw an change in proportion of people using the city

Daytime (8am to 6pm) -> Evening (6pm to 9pm)

Night time (9pm to 8am) unchanged

Modelling Footfall with ML

Explanatory factors

Bank Holidays

What type of bank holiday?

School and University Holidays

The weather

Mean temperature, wind speed and rainfall

Day of week

Many others (not tested yet...)

Train prices, car parking availability, business opening times, etc.

Modelling Footfall with ML

Machine Learning Methods

Attempted a number of different methods

Mainly neural networks and random forests

Random forest was most accurate

Modelling Footfall with ML

Feature Importance

Modelling Footfall with ML

Predictive Analytics

Modelling Footfall with ML

Conclusion

Still needs some refinement, but otherwise a useful tool

Potential for prospective analysis as well as forecasting

Not useful for more nuanced analysis

How should roads be re-configured to encourage pedestrians?

Where are most visitors coming from?

How have patterns of use in the city changed?

For this we need more detail about individual movements...

Improved policy to mitigate pollutant and inactivity related health burdens through new big data

Aim: Take new 'Track and Trace' (T&T) data generated from mobile phones to support new policies to:

Reduce the disease burden of pollution

Encourage active travel

Lead by the Institute for Transport Studies, in collaboration with Newcastle City Council and funded by the ESRC

A more nuanced measure of population flows?

Go Smarter

Smart-phone app built in collaboration with Newcastle City Council

Tracks peoples' journeys

Detects when the user is moving and estimates mode of travel

Rewards for using active / sustainable modes of travel

Aim: Demonstrate how the linking of high-resolution location data and other databases / models can support better policy making

Disease Burden of Pollution

Collaborating with the Newcastle Urban Observatory who are sensing the urban environment

Aim: use T&T data to model urban flows and identify the most serious pollution hotspots.

Data Caveats

T&T data are

High resolution (spatio-temporal)

Abundant

BUT:

How representative of the wider population?

Abundant enough?

Other 'Big' Urban Flow Data

Social Media

Urban Flow Data

Mobile Communications

Large population coverage

Private, unknown methodology, privacy concerns, coarse resolution (?)

Urban Flow Data

Geo-Apps

Smart-phone apps that capture movement / location are becoming ubiquitous

Great potential for understanding (some) urban dynamics

Urban Flow Data

Agent-Based Modelling (ABM)

Autonomous, interacting 'agents'

Model phenomena from the 'bottom-up'

Advantages:

Modelling complexity, non-linearity, emergence

Natural description of a system

Bridge between verbal theories and mathematical models

History of the evolution of the system

ABM Example - Burglary

ABM Problems

1. Computationally Expensive

Not amenable to machine-led calibration

2. Data hungry

Need fine-grained information about individual actions and behaviours

3. Divergent

Usually models represent complex systems

Projections / forecasts quickly diverge from reality

3. Divergence

Drawback with the 'typical' model development process

Waterfall-style approach is common

Calibrate until fitness is reasonable, then make predictions

But we can do better:

Better computers

More (streaming) data

Methodological gap

Ensemble Kalman Filter - Basic Process

1. Forecast.

Run an ensemble of models (ABMs) forward in time.

Calculate ensemble mean and variance

2. Analysis.

New 'real' data are available

Integrate these data with the model forecasts to create estimate of model parameter(s)

Impact of new observations depends on their accuracy

3. Repeat

Ensemble Kalman Filter - Basic Process

Experiment with an EnFK

Very simple ABM

People walking along a street

Every hour, x people begin at point A

CCTV Cameras at either end count footfall

Some people can leave before they reach the end (bleedout rate)

Aim: Estimate the number of people who will pass camera B

Hypothetical 'Truth' Data

Use the model to first generate a hypothetical reality

(Preliminary) Experimental Results

(Preliminary) Experimental Results

Forecast and analysis are barely distinguishable

Virtual observations are closer to 'truth' than the analysis :-(

This is probably due to the degree of randomness in the model

EnKF estimates the model parameter (bleedout rate) accurately :-)