How to check a dataset is safe to publish, and why removing the names is not enough
A file with the names removed still carries postcode, date of birth and sex, and those three together pick most people out of a population on their own. Here is what k-anonymity actually measures, what it costs to raise it, and why a comfortable k can still give the whole answer away.
Put the file into CROWD before you publish it. Tell it which columns somebody could already know — postcode, date of birth, job title, the ones that identify rather than reveal — and it reports the smallest group in the data, how many rows are unique on those columns, and what each available generalisation would do to the figure, measured by actually doing it. Nothing is uploaded and it works with the network off.
Removing the name is not anonymising
The dataset that taught everyone this was a release of hospital records with the names taken out. It still carried postcode, date of birth and sex — and those three together are enough to pick most people out of a population, because almost nobody shares all three with anybody else nearby.
Latanya Sweeney's analysis of 1990 United States census data put the share of the population uniquely identified by that combination at 87 per cent. A later re-analysis on 2000 census data put it closer to 63. The argument about the exact figure is beside the point: both numbers are catastrophic for a file you were about to put on a website.
The columns that do this are not the sensitive ones. They are the boring ones — the ones an adversary can already look up about a person they know.
What k-anonymity actually measures
Group the rows by the combination of those identifying columns. The size of the smallest group is k. If k is 1, at least one row in your file is unique on that combination, and anybody who knows those facts about that person has found their row and can read everything else in it.
k = 5 means every row hides among at least four others. k = 20 is comfortable for a public release. k = 1 is not a release, it is a disclosure.
The figure reported here is the minimum, not the average, because the average is worthless: a file where almost every group has fifty rows and one has one is a file with a k of 1, and reporting "average group size 49" would be a way of not saying so.
Generalising, and the cost of it
The fix is to make the identifying columns less precise until the groups get big enough. A postcode can go to sector, to district, or to area. A date can go to the month, the year, or the decade. Every step raises k and destroys some of the analytic value of the file, and the whole judgement is about where to stop.
So each option is shown with the k it would produce, worked by applying it to your data rather than estimated. Usually one column does most of the work, and finding out which one that is turns an argument into a decision.
k is not enough on its own
This is the part almost always left out. A group can satisfy k-anonymity and still give the answer away.
Suppose k = 10, so ten people share a postcode district and a birth year. If all ten rows carry the same value in the sensitive column — the same diagnosis, the same outcome, the same status — then finding the group is finding the answer. You never had to identify which row was theirs.
l-diversity counts how many different sensitive values appear inside each group. Where a group is uniform, the file leaks that attribute regardless of its k, and the page says so separately.
What this does not protect against
Being straight about the limits matters more here than in most places, because a green number is exactly the thing somebody will quote in a meeting:
- A second release. Two files that are each fine can be linked. Publishing the same population twice with different generalisations can undo both.
- Outside knowledge you did not model. k is measured against the columns you nominated. If an adversary knows something you did not list, your k is not their k.
- Small populations. In a village, a postcode district and an age band can still be one person, whatever the table says.
- Free text. A notes field will re-identify people no matter what the structured columns say. It should not be in a release at all.
k-anonymity is a floor and a discipline, not a guarantee. It stops the accident that happens to organisations who thought deleting a name column was enough.
The short version
| The job | Here | Notes |
|---|---|---|
| Find the smallest group and the unique rows | Yes | The minimum k, not the average |
| Show what each generalisation would do | Yes | Worked by applying it, not estimated |
| Postcode to sector, district or area | Yes | And dates to month, year or decade |
| Flag groups that are uniform on the sensitive column | Yes | l-diversity: k alone is not enough |
| Keep the file off a server while you check it | Yes | The point; it is the file you have not cleared yet |
| Anonymise the file for you | No | It measures; the decision is editorial |
| Account for a second release | No | Linkage across files is not modelled |
| Protect a free-text notes column | No | Nothing can; it should not be released |
| Tell you k is safe | No | Safe depends on the population and the adversary |
Why it matters that it stays on your machine
The file you are testing is, by definition, the one you have not yet decided is safe to release. Uploading it to a service to find out whether it is safe to upload is a circle nobody should be standing in, and for a public body it may be the disclosure the whole exercise was meant to prevent.
Load the page, turn the network off, and check it there.
Questions people ask about checking a dataset before release