Hackathons as gateways to more and better open data

Posted: August 22nd, 2011 | Author: | Filed under: Inspiration | Tags: , , , , | No Comments »

There is a piece up on O’Reilly Radar by Andy Oram about the sustainability of applications built during hackathons. I am involved in Hack de Overheid and we have organized (Apps for Amsterdam) and still are organizing (Apps for Noord Holland) several hackathons and I thought it would be good to add our experiences to the fray.

First: I do not agree with the premise that most apps created in government challenges are quickly abandoned. I have not done a tally of our Apps for Amsterdam contest, but the completeness and polish of most apps submitted was impressive. I still use several of the apps from that contest regularly. Snelstepontje.nl for finding out which ferry to take is a godsend just to name one.

Maintenance is indeed an issue. It is my personal experience that if the app is deployed to a suitably robust platform (Google App Engine is a notable one), it may continue to run unsupervised for many years.

But yes, I do have my own doubts when it comes to the sustainability of apps from app contests as I have stated in my review of Apps for Amsterdam.

Data quality is the largest issue on all levels and it needs to be addressed. From gathering data, to publishing it, to responding adequately to issues. Most datasets that are released for contests are not of the highest quality due to time constraints. And after the contest is over they are seldom kept up to date by the publishing office. When it comes to sustainability, government should first turn to itself and start releasing their data in a way that is sustainable.

Besides releasing the data in a proper format, a very important consideration is the licensing. Re-using data should happen under conditions as liberal as possible (CC0 preferred) as not to deter companies from investing in using that data.

But even then creating apps that are successful and sustainable at scale may be too lofty a goal. Productizing apps in a professional way implies conceiving, building and expanding a startup company. If one or more such initiatives come out of a hackathon that may be called a resounding succes. But what of the rest?

Well, communities of practice are built on exactly that: practice. Data does not overnight become readily at hand and usable. It takes a lot of hard work from all of us.

Having organized several hackdays we are seeing an increase in number of people attending and their proficiencies as well as a wider awareness of the possibilities of data in journalism, government and politics. Those are exactly the things we need if we want to make open data (and not just applications) the foundational fabric of our information society.

Video of The Rising Tide of Data

Posted: November 18th, 2010 | Author: | Filed under: Events, Talks | Tags: , , , | No Comments »

Watch Alper give a broad sweeping introduction on the field of data at a recent Mobile Monday Amsterdam:

Video for Civic Duty

Posted: November 12th, 2010 | Author: | Filed under: Talks | Tags: , , , , , , , , , | No Comments »

The previous video for Civic Duty in a Hyper-Connected World was somewhat poorly accessible because it is locked in a HVA courseware environment. The Club of Amsterdam has graciously provided the raw video which we converted and uploaded to Vimeo.

Sound is not completely in sync, but this should serve as a nice narrative accompaniment to the slides posted here before:

Civic Duty in a Hyper-Connected World from Alper Çugun on Vimeo.

The Rising Tide of Data on Mobile Monday

Posted: November 10th, 2010 | Author: | Filed under: Events, Talks | Tags: , , , , , | 1 Comment »

“The Rising Tide of Data” was a more pared down, less activist, introductory version of “Civicy Duty in a Hyper-Connected World” presented by Alper and it was well received in a solid program by Mobile Monday Amsterdam on Data.

Esther Gons (also pictured below) does visual representations of complicated concepts and she did this sketch of my presentation:

Click for a bigger view and her take on the event.

DaphneChannaHorn-MOMOAms-18 (29 van 59)

Annotated slides and video will be available shortly.

The Rising Tide of Data on Mobile Monday

Posted: November 3rd, 2010 | Author: | Filed under: Events, Talks | Tags: , , , , | No Comments »

Alper will be giving a brief data-heavy talk titled “The Rising Tide of Data” at Mobile Monday Amsterdam #18. More to follow.

Video of Club of Amsterdam Presentation

Posted: October 28th, 2010 | Author: | Filed under: Talks | Tags: , , , , , , , , | 1 Comment »

The video of my recent presentation for the Club of Amsterdam (slides and notes) has been posted to the HVA site. Find it here (start at index: 44:30): The Future of Hacking

Future of Hacking1/2

Working on releasing it in more formats.

Opinion piece on public transportation data in Dutch newspaper NRC.next

Posted: October 28th, 2010 | Author: | Filed under: Publications | Tags: , , , , , , , , , , | No Comments »

Yesterday Dutch daily newspaper NRC.next published an opinion piece I’d written for them concerning Dutch public transportation information. Alexander Klöpping had written a piece before and mine was to be a more specific follow-up on a certain kind of information: transit.

The situation is that Dutch transit information is controlled by a single private entity who are not sharing any of their data. The Dutch government plans to write a tender for a public data warehouse (to be called: NDOV) to be built with all transit information in it. Now this is both a threat for the private institution currently exploiting the data and an opportunity for the Netherlands to make a —much needed— jump forward in the realm of transit information. Which makes this an interesting turning point.

After making the altogether clear and well-known argument for open transit data, I propose three hard requirements such a data warehouse should fulfill. Here in English:

  1. All public transit data: journey plans, live locations, departure times and (un)planned mutations in the NDOV have to be readable both for humans on a website and for computers via an API.
  2. All this data has to be freely accessible to everybody without limitations.
  3. There always has to be a high quality journey planner but others have to be allowed to build their own or build on top of it.

Doesn’t sound too unreasonable, does it?

Full text in Dutch on Alper’s personal weblog for those interested.

Slides for Civic Duty in a Hyper-Connected World

Posted: October 15th, 2010 | Author: | Filed under: Talks | Tags: , , , , , , | 6 Comments »

Yesterday Alper gave the presentation on improving our environments using data and hacking for the Club of Amsterdam. Here are the slides from that presentation with notes:

I am intendant at an Amsterdam based technology and design firm called Monster Swell. We’re occupied with creating visualizations and other systems out of data. Some examples of our work:

We landscape silence.

We map discontent.

We create atlasses.

The issue I wanted to bring forward on this event about the future of hacking is about how we can hack our ways to a better future, and how we in fact MUST do that, because if we won’t change our future for the better, somebody else may do it for the worse. Hacking is in fact a civic duty.

First I will start with a background of the context we’re going to do this all in.

A Hyper-Connected World. Fantastic illustration (CC-by-nc license) from Tom Coates’s dConstruct talk.

Insert all your old web 2.0 stapels here: social networking, the web of creation. Be it on facebook, twitter, last.fm or wikipedia.

And we could use these to talk with each other about the things that occupy us, though most (not all) of this was very virtual.

Now the world starts to talk back to us using sensors, fields, smarts, wires and wireless

“That which primarily conditions choice and action in the city is no longer physical, but has become a matter of the invisible and intangible overlay of networked information.” —Adam Greenfield at LIFT

On the one hand everything’s going to be crammed full with sensors. The dutch word: ‘volgesensord’. A terrific sensorification.

Types of sensors: audio, movement, orientation, video, RFID, IR, etc.

Personal: everything you can cram into a cell phone

De-personalized: cc-tv, car/cycle-loops, transit gates

photo license: Creative Commons by-nc

Many of these sensors are personal and embedded in our most personal devices. The iPhone alone has a great many of them.

Wireless in the world from timo on Vimeo.

But many others are all around us with their own sensor fields and connections to each other and the network, permeating everywhere through our lives without us even knowing it.

“Wireless in the world” video by Timo Arnall

Pachube by Usman Haque is a network backbone for these kind of sensors. It is one of many visible switching stations of the data in/out points that you can use to link up with some of the sensors in the world.

So the world can sense data, but it can also show data back to us. Displays themselves by feedback and conditioning already change people’s behaviour.

Each and every representation we make is subjective, but it is a connection between the real life territory around us and the map we deduce from it (Tom Coates).

Reality is made up of the built environment, but more and more layers of information are getting added on top just like on this Pretty Map (by Stamen), which is pretty pretty.

Flickr Shapefiles for cities and neighborhoods
Urban areas from Natural Earth
Road, highway and path data from OpenStreetMap

GPS sensors in every train, added to the display, knowing where a transit object is at any given moment will influence your behaviour.

This got built by a couple of people on ScienceHackday in London in an afternoon. In the Netherlands we are prohibited by our transit agencies from building something similar.

This is Nuage Vert.

We can map these variables away from the computer screen and onto reality itself.

This is a particularly inspired example of that: “the vapour emissions of he Salmisaari power plant in Helsinki will be illuminated to show the current levels of electricity consumption by local residents.”

Remember Guid Raid 11/20/09 in Orgrimar from Daniel Petersen on Vimeo.

This is another kind of territory in World of Warcraft with a highly specialized and optimized display, made to help you to keep certain parameters within certain bounds and win the day for your team.

LayAR is an Amsterdam based company making great strides in adding an additional layer on top of reality and turning the above World of Warcraft video into reality quicker than I am really comfortable with.

Augmented (hyper)Reality: Domestic Robocop from Keiichi Matsuda on Vimeo.

The latter half of the 20th century saw the built environment merged with media space, and architecture taking on new roles related to branding, image and consumerism. Augmented reality may recontextualise the functions of consumerism and architecture, and change in the way in which we operate within it.

A film produced for my final year Masters in Architecture, part of a larger project about the social and architectural consequences of new media and augmented reality.

Just like the shipping container revolutionized transportation, we get to interact with the world because the roads have been cleared. The network and its basic transport has been standardized. Everybody speaks HTTP, APIs are roughly figured out, devices are talking to each other.

Fucking shipping container cliché (but, hey it works!).

photo license: Creative Commons by-nc-nd

We are trying to work this equation. Getting things to act in an interesting and useful way is one of the biggest design challenges of our time.

Wherein I try to illuminate some stuff about our time.

We are inbetween something, an interregnum of sorts. I can’t really prove this, but I guess more of you may feel that we are at an inflection point at times where great changes are afoot and nobody can say with any amount of certainty whether the future will look better or worse or make any meaningful predictions about it.

Most science fiction writers can’t write a future anymore because the present is too shaky, or if they do it’s usually rather dismal like this Douglas Coupland piece.

Gramsci said: “The old is dying and the new cannot be born.  In this interregnum there arises a great diversity of morbid symptoms.” “now is the time of monsters”

Referenced via Jon Taplin’s state of America.

Combine that with the Shirky principle: These old dying institutions are with their last breaths trying to preserve the problems they are solving while they may not be relevant anymore in a networked world.

photo license: Creative Commons by

Dam Square, Amsterdam

Public space built around physical infrastructures was the meeting ground for people. This is Dam Square, this used to be a center of not only trading, but of knowledge exchange and power. How very much unlike it is today. Smart people would go here, now it’s just tourists and other people without taste that convene on this square.

When the city becomes an increasingly virtual entity composed of networked services, these publics will form around shared issues of concern, along non-traditional lines, and different places.

See the presentation on Digital Infrastructures by Martijn de Waal.

photo by Shane Gavin, license: Creative Commons by

The information we have at our disposal and the speed of media have lead to a partial crumbling of traditional authority not only in politics but also in law, medicine, media and any field really.

photo license: Creative Commons by-nc

The world is getting increasingly complex and interconnected

So with this hyperconnected world and the changes in society, what can we do?

Video: New York Talk Exchange

These issues mean that this is the time for hacking. We cannot lean back and trust the future will be better than the past. We need to make it so.

Hacking is using things for unintended consequences to improve the current situation. A better word for ‘innovation’ if ever there was one.

A bottom up approach each person has their preferences, their needs and their agency and they try to create better choices and reduced opportunity costs for themselves.

This is not me, but some fat dude on Flickr (though of course you only have my word for that). An example of people who are measuring and hacking their lifes are the Quantified Self group. A bunch of grownups who are sensorizing their lives and measuring EVERYTHING to be able to spend their time and energy better.

photo by Michael Verhoef license: Creative Commons by-nc-nd

Street Hacks, The street finds it own use for things (William Gibson). Creative adaptation. People modify the offerings of companies and governments for real life use already. How can they do the same for the information offerings provided?

How will information and interaction become a hackable commodity? What tools do we need to build for people to be able to create their own solutions?

photo by Jan Chipchase, license: Creative Commons by-nc-sa

Privacy does not equal secrecy. Privacy is the function where somebody is in control of their personal information and can see what happens to it. It is very much more a perceptual quality than it is a hard one.

Facebook saying that people can /control/ their information does not help anybody if the GUI and its side effects are too complex (still) for people to do so in a meaningful way.

Information works for those that are able to use it. Enabling literacy and offering information in a palatable way to people is a necessary step for everybody to be able to become a part of information society. Too much of technology as it is right now is magic for too many people.

A top down approach where we try to design a society that bypasses the tragedies of the commons and the lowest common denominators?

We need to create an increase in engagement and participation, judiciously using transparency and design to create better living conditions.

There are loads of initiatives and policy makers are desperate to include more people into the process but most efforts are not much more than lip service for a variety of reasons. We need to figure out better how to include more people.

Obligatory Jane Jacobs quote

This is a Maneki Neko, which is a cat and the eponymous short story by Bruce Sterling.

It’s a great read and a rather ideal solution to what a digital government should/could look like if everything worked perfectly.

Our government shows itself to be more than capable in wasting hundreds of millions of €s on failing IT projects without showing much transparency or innovation for it. Especially in this period of crunch, this is simply not good enough anymore.

Systems need to work, they need to be resilient and user friendly up to much higher standards than we have had thusfar. How we will do this is an important but open question.

Lessig has a point here and that faith is bearing awfully thin as it is in our part of the world.

Right wing parties in Amsterdam are right now interested in opening up budget data. Normally they aren’t this in favor of open data initiatives but their motivation here is simple and clear: they want to cut budgets, and to campaign to cut budgets, you must first show that there are budgets. Transparency helps them and if there’s no informed response, they will get away with it.

photo by Joi Ito, license: Creative Commons by

A bunch of examples how we are using data and representation to change our environments both from the top down as from the bottom up.

This is my personal map of Amsterdam on Weeplaces. I can see where I’ve been regularly and what my friends’ preferences are. This uses Foursquare for the location data, but that could be Facebook or Hyves or any other social checkin service.

The act of checking in creates a personal, local and global taste map of the area. All these checkins in aggregate give you views of the area, enable you to create personalized maps.

MIMAQ: Mobile Individual Measurements of Air Quality
project by Digitale Pioniers + WAAG

Democratizes the measurement tool so that everybody can measure themselves instead of relying on the measurements provided by some institution (RIVM). Also the feedback and presentation of measurement values is a lot better (a lot more gamelike) than those normally provided to us by government institutions.

Verbeter de buurt, the Dutch version of Fix My Street

Verbeter de buurt is a way of linking your mental model of what is wrong in your area with your local council and using the web to both campaign for your issue and publicly shame your council to fix stuff. A great way of how a subjective map can be used to change reality.

Chromaroma Visualisations from Mudlark on Vimeo.

A montage of visualisations generated from the travel data on the live alpha version of the game Chromaroma

Chromaroma, your ambient actions in London transit are a game for you and your team. Top down game mechanics can influence overall transit behaviour.

Get excited and make things. It is now easier and cheaper than ever before to organize, act and build/create stuff. So why don’t we go ahead and do that? Society is malleable.

I’m going to show some examples just how malleable things are:

Thanks Matt Jones for a visual and a Zeitgeist (image under Creative Commons by-nc-sa license).

A project by a friend trying to convert Times Square into art: Times Square to Art Square (new website and a party on Monday!). Justus Bruns had this idea at some point and because he was surrounded by the right people, they told him: “Why don’t you go do it.” Which he is now doing by sheer conviction and the help of a large group of people.

Another piece by Adam Greenfield on community development and the Network Revolution blog by Ernst-Jan Pfauth.

This is Newspaper Club (they just started shipping internationally). Russell Davies says: 
“There are massive bits of infrastructure lying around unused.”

The old is not going away any time soon, but we can use it in new ways. Repurpose existing systems in interesting ways by leveraging the network or similar effects.

This is a reprap. It is a special kind of 3D printer, one that has the plans and the facility to print out itself.

There are more fablabs and repraps in the Netherlands per capita than anywhere else in the world. A rare example of innovation subsidies gone to the right spot.

We have the tools and the means, the only thing remaining is to actually do it. Hacking is our civic duty. We can’t wait for hulks of institutions. They’ll only believe us if we show them it’s possible anyway.

And we have to be the ones who do it to be able to determine what the results are.
“What it is… is up to us.” —Howard Rheingold

Thanks to Adam Greenfield for opening up and stimulating much of the (ethical) crossover thinking between the real world and the online world.

Also credits to this deck by Chris Heathcote.

Presentation under a Creative Commons Attribution Non-Commercial ShareAlike license.