Written by: Mark Chapman

Written by: David Elston

Written by: David Elston

Written by: David Elston

Written by: David Elston

Written by: Danfox Davies

Written by: Danfox Davies

Written by: Danfox Davies

Written by: Mark Chapman

Written by: Loz Kaye

Written by: Andy Halsall

Written by: Adrian Short

Written by: Loz Kaye

Written by: Loz Kaye

Written by: Andy Halsall

Written by: Andy Halsall

Consequences of the IP Bill for You and Your Internet Service Provider

Well, like it or not (and I don't) the Investigatory Powers Bill has received royal assent and so is something that Internet Service Providers (ISPs) have to take seriously. But what does it all mean for you and your ISP?

Internet Connection Records

The new bill requires that Internet Connection Records (ICRs) are retained by communications service providers. The intention is that government agencies shall be able to look at the retained data subject to due legal process allowing law enforcement agencies to attribute illegal activity on the internet to a person in the real world.

The Government has been famously coy in defining what an ICR actually is. They say things like, "Internet connection records (ICRs) are records of the internet services that have been accessed by a device. They would include, for example, a record of the fact that a smartphone had accessed a particular social media website at a particular time." And they give examples of usage such as, "Internet Connection Records when used with Internet Protocol (IP) resolution data would allow law enforcement agencies to trace the individuals who accessed [particular] images."

The Government is also are quite keen to say what they are not: "ICRs do not provide a full internet browsing history. The ICRs do not reveal every web page that a person visited or any action carried out on that web page."

But this is all very unclear. Does the ICR stop at noting that you visited www.blogspot.com, or does it go as far as distinguishing visits to mamarihanna.blogspot.com and womenhealth85.blogspot.com?

Who Has to Store ICRs?

In other words, what constitutes a "communications service provider"? It is pretty clear that at least a primary objective is that an ISP will be expected to retain ICRs. That is, the company that provides broadband or mobile access to the Internet for individual subscribers. And that will cover a substantial proportion of all access to the Internet. But it doesn't cover everything and, since the intent is supposed to be to find the criminals, we may assume that it will be necessary for other providers of Internet access to also store ICRs.

So who else might be involved? At one end of the spectrum, it is possible that the Government will want to interpret the bill to mean that transit providers (those network operators who do not sell connectivity direct to end-users, but whose services connect together other networks) are also expected to retain ICRs. This might be limited to domestic traffic (that is, Internet connectivity where at least one end of the connection is within the UK), but it might be extended to apply to all traffic that passes through the UK.

At the other end of the spectrum, Wi-Fi hot spots (cafes, libraries, etc.) could quite easily be counted as communications service providers since people go to them to get Internet access. In some countries, it is a legal requirement that these providers collect personal details about users - usually this is achieved through a log-on screen where anyone could enter absolutely anything they want, and so it doesn't really serve any purpose. In this case, collection of ICRs would seem to have no value at all except to establish the physical location from which an action was performed. It has been suggested that retention of the media access control (MAC) address of a computer or phone (a unique identifier assigned to network interfaces of computers) could form part of the ICR and provide some correlation back to the specific device and hence to its user, but most modern devices allow the user to change the MAC address at will, and some devices (like smart phones) automatically vary the address.

Perhaps an even more confusing situation arises for companies that operate wireless or fixed networks for their employees and so provide them with Internet access. While their ISP may be collecting information in turn, the details of which user did what will be hidden because most such networks (just like home networks) use network address translation (NAT) that maps multiple network users onto a single external IP address. Will such companies be required to maintain ICRs or store records at the NAT device so that the Government can put them together with the ISP's ICRs to work out who was performing which actions?

Community networks are a very special type of ISP. Usually run on a shoestring and not-for-profit, a community network provides connectivity and Internet access for a local community and is usually operated by that community. Such networks commonly lack financial and equipment resources to perform anything other than basic services, and the network operators are quite often untrained volunteers struggling to do enough to keep the network up. Such networks will struggle to meet demands to retain ICRs.

A final category of communications service provider may be those companies who offer services that are accessed over the Internet.

These services could range from email to search engines. They could include web hosting and electronic trade and payment sites.

They would very probably include social media and content sharing sites. And, of course, they could include domain name system (DNS) servers that we all use when attempting to visit web sites from our browsers.

In practice, then, the term "communications service provider" is scoped so widely as to cover anything and anyone providing electronic communications, Internet access, or services on the Internet. There is no reason to suspect that this definition was not deliberately broad so that the Government can apply the bill at their own discretion at any time.

What do Communication Service Providers Really Have to Store?

The details of what an ICR is have been left unclear. Given the demands for explanation while the bill progressed through Parliament we may conclude that the vagueness is deliberate and exists to allow the Government to modify the definition of what they want collected as needs develop.

At the moment, the Government simply says, "ICRs are records of the internet services that have been accessed by a device," which is completely open ended. Some qualification is provided by an example that they provide: "a record of the fact that a smartphone had accessed a particular social media website at a particular time," but the details here are suspect and presumably associate the SIM in the phone with the access to the website.

Further details can be deduced from the three stated purposes:

  • To identify the sender of a communication. This could be used to locate the particular device from which an illegal image was uploaded to a website at a particular time.
  • To identify the communications services a person is using. This would allow the police to determine whether a missing person was [had been] using a particular smartphone app or social media website prior to his or her disappearance.
  • To determine whether a person has been accessing or making available illegal material online.

But this remains pretty unclear. What exactly is an ISP supposed to store?

The Government attempts to provide a little more information saying:

  • Internet connection records are records captured by the network access provider of the internet services with which a uniquely identifiable device interacts.
  • It will involve retention of a destination IP address, but can also include a service name (e.g. Facebook or Google) or a web address (e.g. www.facebook.com or www.google.com) along with a time/date.
  • It could never contain a full web address as under the law these would be defined as content.
  • You may be able to see that a person has used, google.co.uk or facebook.com but you would not be able to see what searches have been made on google or whose profiles had been viewed on Facebook.

At this point it looks like a relatively simple record of IP address and domain name, but it should be noted that this information does not come close to providing the details that would be necessary to meet most of the requirements for addressing crime, terrorism, and paedophilia that the Government has stated. So there is probably going to be further information that has to be stored.

There was a suggestion that the Government would only be interested in access to specific services, but that would require the communication service provider to analyse data, and the intention (as currently stated) is both to protect the service provider from having to parse the communications, and to protect the users from that data being visible to anyone except from the Government agencies. So it looks like all data has to be stored.

How Long Do They Have to Store ICRs For?

Part of the philosophy to capturing ICRs is to be able to perform a historical analysis of Internet activity. Who accessed a web site in the last six months? What is the pattern of an individual's social media activity? What is the chain of communications and connectivity leading up to an event?

That means that ICRs (unlike a phone tap) need to be collected speculatively and retained for future examination. Figures of 12 months have been mentioned, but no specific limits have been stated. Obviously, the storage time is important in understanding how much data has to be stored.

How Much Data is That in Total?

It depends how the data is recorded and stored. And obviously it depends on what an ICR is and for how long it is retained.

Let's assume that an ICR contains:

  • the source IP address (16 bytes in IPv6)
  • the destination IP address (16 bytes)
  • the source and destination port numbers (2 bytes)
  • a payload protocol identifier (1 byte)
  • a timestamp (8 bytes)
  • customer/subscriber identifiers, such as phone number, SIM, etc. (16 bytes)
  • URL or domain name of service visited (32 bytes - average)
  • service indicator (4 bytes)
  • assorted additional data (32 bytes)

That makes about 150 bytes for each ICR.

Now, suppose a user is browsing quite actively. Maybe they reasonably visit a new web site every 5 minutes over a two hour period. That's 24 ICRs making 4KB of storage. Or 1.5MB per user per year.

Of course, some users will use the Internet more actively and for longer periods meaning they generate far more data. While other users will only be on line from time to time.

So maybe 1MB per subscriber per year is a reasonable working figure.

But each subscriber hides multiple devices as every home has multiple computers and averages more than one person.

So perhaps this scales to 3MB.

Now a middle of the road ISP like TalkTalk has about 4 million customers.

So we're talking about 12 Terabytes which doesn't seem unreasonable.

But these numbers are just guesses. When you visit some web pages they pop up a number of frames with adverts and other content each sourced as a separate web session. Maybe more information needs to be retained. Maybe users are far more active. What about DNS transactions, email reading, database transactions, cloud computing, UDP-based protocols? In fact, the amount of data could be very many orders of magnitude larger meaning that the 12TB is a lower bound and the upper bound could be thousands of times larger.

In fact, a reasonable guess would be to multiple by between one and ten thousand taking the upper bound to 120000TB.

For reference, a 4TB drive retails at about 100 pounds. So TalkTalk could expect to spend around 3 million pounds to store this data.

Double all these figures if the data must be protected against device failures, fires, etc.

Is the Data Secure?

The communications service providers are required by law to keep the data securely. The Government says, "Retention of this data will be subject to stringent security requirements, including audit by the Information Commissioner." We should recall, however, that existing data protection regulations require that ISPs keep their customer records securely, but despite that there are frequent leakages (usually, but not always resulting from hacking) of customers' accounts and payment details. If an ISP cannot keep the billing information safe, why should we assume that the ICRs will be safe?

When Do They Have to Act?

It appears that communication service providers do not need to start recording ICRs at once. The Government says: "A data retention notice for internet connection can be placed on a CSP by the Secretary of State, where necessary and proportionate." That appears to say that until a notice is served, the service provider can continue with business as usual. Of course, when they are served with a data retention notice, they may find themselves unable to immediately comply if they don't have the storage and means to capture ICRs in place.

On the other hand, when the notice is served it may require the CSP to do pretty much anything.

When and How do They Deliver Up the Data?

Some ISPs had hoped that they would simply deliver up the ICRs to a central agency where it would be stored and from where it could be distributed according to permissions enshrined in law, approval by designated authorities, and oversight by the Investigatory Powers Commissioner. But currently it doesn't look like that. It appears that communications service providers will be required to store the data and serve it up on request.

There is currently no description of how the data must be delivered (on what medium, in what format, and with what security).

Will We Know What is Being Done?

No, not really. One of the provisions of a data retention notice is that the communications service provider shall not disclose that it is retaining ICRs, nor if and when it delivers them to a Government agency. That means we either have to assume that all of our details are always being recorded, or we should use an ISP that makes a public commitment to close down rather than collect ICRs.

Who Pays?

The Government has said...

In the UK, government policy is to fund 100% of the reasonable costs incurred by CSPs in complying with communications data retention notices. This means that CSPs are not financially disadvantaged by compliance and are not incentivised to pursue lowest cost solutions. This arrangement enables the Secretary of State to achieve an appropriate balance between operational benefits and the cost of CSP compliance solutions.

In theory that means that:

  • No one has to spend any money until served with a data retention notice.
  • No data will be retained until suitable hardware is in place after a data retention notice has been served.
  • Communications service providers will be able to claim back the costs of equipment and staff.

And who pays in the end? Well, of course it is the tax payer. And possibly, if we scale up from the example figures above, the cost could be as small as 20 million pounds. Although it seems likely that multiple providers might end up storing the same data.

Why is This All so Confusing and Chaotic?

The basic answer to this question is that the Government is trying to legislate for the impossible. For example, the Government says, "Without the retention of ICRs, resolving an IP address back to a single user will often not be possible as multiple users may be associated with that IP address. ICRs therefore provide the unique identifier to distinguish between different users of a shared IP address." That is an entirely reasonable objective, but is completely impossible to achieve with today's technology because in a home network all devices share a common public IP address and their identifiers within the network vary depending on when they power up and join the network.

It is fair to say that the Government consulted widely before passing the IP Bill. But it could not be said that they listened to the feedback they received. When ISPs and other experts on the Internet pointed out the considerable problems with their plans, the Government chose instead to listen to the demands of the security services tasked with "keeping us safe" and "clamping down on crime."

We could draw similarities with the 1897 bill in Indiana that tried to legislate for a neat and useful method to square the circle (a highly desirable function in engineering and surveying) that, along the way, declared that the value of pi was 3.2.

It might also be interesting to make comparisons between the current state and use of the Internet, and the way that printing and pamphlets were used by the Levellers in the 1640s.

What Should We Do and Why?

While we may fully support the Government in its concern to protect us from terrorism and to shut down paedophilia and other organised crime, we do not have to accept a situation where our personal Internet history is gathered, stored, accessed by any number of government or quasi-governmental organisations, and probably leaked to organised crime. In fact, we should be extremely worried about government agencies that are able to know whether and to what extent we engage in activities designed to oppose the will of the Government. And in a world where ransomware is an escalating Internet problem we should expect our data to be a high value target for criminals seeking to blackmail for money or to arrange social-engineered access to secure resources - just because what we do on the Internet is legal does not mean that we want our family or employers to know about it. Don't forget in all this that even the largest ISPs don't have a good record of keeping our private (e.g., billing) data confidential.

So there are some basic things we can do. Some of these things make our data more secure and help to keep it private. Some of them just make recording the data harder and more expensive. While I am under no illusions that our Government is currently willing to spend unlimited amounts of our own money to gain access to our private data, if we create a high entry cost, it may be that a rethink will be required.

It is worth noting that if you are a criminal or a terrorist, you are probably doing all this and more already.

Here is a brief and non-exclusive list of things you could do:

  1. Use HTTPS. If you operate a web site make sure it is available using the secure web protocol, HTTPS. If you visit web sites, prefer those that use HTTPS.
  2. Use Encryption
    1. Encrypt your hard disk. This is available in most modern operating systems and doesn't slow down performance substantially.
    2. Encrypt your emails when you send them to friends, and also when you store them. Consider using an off-shore secure email provider such as ProtonMail.
    3. Look very carefully at where your encryption software was produced! Is it fully OpenSource so that it has been carefully reviewed? Was it made in a country you consider unlikely to have inserted backdoors?
    4. Consider using Encryption on your Voice, Instant Messenger, and SMS communications. Look at services like Signal from Open Whisper Systems.
  3. Use a virtual private network (VPN) to secure the traffic from your computer into the Internet. But be aware that a VPN operator in the UK will probably be classed as a communications service provider under the terms of this bill, so you may want to consider using an off-shore VPN or even Tor.
  4. Use UDP-based tools rather than conventional TCP-based approaches. Although applications may still be session-based, using UDP as a transport mechanism makes it very much harder (or at least more expensive) to track connectivity and usage. In that context we should encourage the work of the IETF's QUIC working group that is developing specifications for a UDP-based, stream-multiplexing, encrypted transport protocol, and as soon as it is available we should start to use it.
  5. Use non-tracking search engines when you need to search for things on line. Groups like DuckDuckGo claim to not track your searches and to not record your search history. Of course, if they are approached by the national government where they host their servers, they may have to start keeping records and would not be allowed to tell us, but at least you know that your ancient history will not be available.
  6. Be careful how you use DNS. The domain name system is used to resolve names like www.pirateparty.org.uk to the IP address of the hosting server (such as By watching your DNS requests, the Government can get a good idea of your browsing behaviour, and recall that a DNS provider will probably be classed as a communications service provider and required to retain records.
    1. Use off-shore DNS providers. You don't have to use the default DNS service offered by your ISP and you can choose one that is hosted out of the reach of this Government.
    2. Use non-tracking DNS providers. Just like with search engines, some DNS providers (such as Verisign) pledge to respect your privacy and to not track your DNS requests.
    3. Consider using secure DNS (DNSSEC) where it is available. DNSSEC provides privacy for your lookup requests and responses as they flow through your ISP's network, and also ensures that your requests are served by your chosen provider and not intercepted/diverted. The more modern operating systems include support for DNSSEC and the better DNS providers will be happy to accept it. You may discover, however, that your ISP blocks DNSSEC: if they do, you can deduce what that means.
  7. Become active!
    1. Keep yourself informed. Don't just read the popular media, but keep reading and read deep.
    2. Campaign, lobby, and petition. Although the IP Bill is now in place, we can still influence its future. Share information and spread the word.
    3. Tell people how you feel. Communicate with your MP (polite, well-informed letters work best).
    4. Support organisations who research and campaign. Some key groups are the Open Rights Group, the Electronic Frontier Foundation, and the Pirate Party. Join them or make a donation to help their work.

Is that All? What About the Digital Economy Bill?

Sadly the Governments shows no sign of stopping with the Investigatory Powers Bill. The Digital Economy Bill (the DE Bill) has just completed its second reading in the House of Lords and will now continue with its Committee stage.

The DE Bill was described by many of the Lords as "wide ranging" and the very breadth means that it covers lots of important details that need attention (poor mobile phone coverage, availability of terrestrial television services over the Internet, the "digital divide" that means some of the UK population has no access to the Internet, roll-out of "super-fast broadband", teaching of computing in schools, education for the adult population on using computers and the Internet), but also allows the Government to slip in a number far more suspect measures.

For example, they have introduced an "Internet censor". They propose age verification checks that will be used to control who can visit which web sites. And they are looking to relax the terms under which agencies can share data, including not only details of specific individuals but also bulk data.

Many of the plans seem to be poorly thought through. The idea that a teenager might lie about their age or use someone else's identity seems to have been ignored, but it would seem that the solution they might develop would be based on biometric recognition which would be a very nasty development.

Similarly, the fact that a pornographic image might be posted on Twitter seems to have led the Lords to conclude that either every social media posting should be reviewed by the censor before it is posted or that age verification is needed before anyone can access Twitter.

You can find out more about the DE Bill on the Open Rights Group web site.

References and Further Reading

Background Reading

Lobby Groups and Political Parties

Secure Services

Author - Adrian Farrel

Adrian Farrel's picture

More Information

Chat with us


Social Media

Twitter icon Facebook icon RSS icon YouTube icon

Current Internal Elections

Proposal to close the Party - View
Voting opens: Sunday, 20 September, 2020 - 23:59
Voting closes: Sunday, 4 October, 2020 - 23:59
NEC Elections 2019 - View
Voting opens: Monday, 9 December, 2019 - 00:01
Voting closes: Monday, 23 December, 2019 - 23:59
Governor - View
Nominations opened: 11 April, 2019 - 23:59
Nominations close: Thursday, 25 April, 2019 - 23:59
Voting opens: Saturday, 20 July, 2019 - 23:00
Voting closes: Saturday, 27 July, 2019 - 23:00
Nominations Officer - View
Nominations opened: 11 April, 2019 - 23:59
Nominations close: Thursday, 25 April, 2019 - 23:59