Protocols

Internet protocols have more than one model. The OSI model gives us a layered organization:

  1. Physical: transmission and reception of bits on the physical medium.
  2. Link: reliable transmission of frames between two nodes. PPP.
  3. Network: transmission of packets over a multi-node network, with addressing, routing and traffic control. IP
  4. Transport: transmission of data segments between points in a network. TCP, UDP.
  5. Session: session management.
  6. Presentation: translation of the protocols towards an application. MIME, SSL
  7. Application: High level APIs. HTTP, Websockets.

As programmers, we can get involved at different levels of the model:

  • Building protocols based on TCP, UDP using Java Sockets (level 4 and a half).
  • Accessing HTTP (layer 7) web applications, implemented on ports 80 or 443 (secure).

Standard layer 4 and half protocols can be seen in this list.

IP version 4

IPv4 addresses use 32 bits, and have ranges assigned for different purposes:

  • Class A: from 1.x.x.x to 127.x.x.x (subnet mask 255.0.0.0). The range 127.x.x.x is reserved for the loopback.
  • Class B: from 128.0.x.x to 191.255.x.x (subnet mask 255.255.0.0).
  • Class C: from 192.0.0.x to 223.255.255.x (subnet mask 255.255.255.0).
  • Class D: from 224.0.0.0 to 239.255.255.255. Reserved for multicasting.
  • Class E: from 240.0.0.0 to 255.255.255.254. Reserved for research.

Of these, private IPs are considered:

  • 10.0.0.0 to 10.255.255.255
  • 172.16.0.0 to 172.31.255.255
  • 192.168.0.0 to 192.168.255.255

These are the types of routing available:

Unicast: Class A, B and C addresses. TCP and UDP transport.

Broadcast: Host address with all 1's. UDP transport. They do not go through routers, and are received by all machines.

Multicast: Class D addresses. UDP Transport. A machine must listen to receive communication. Available on local networks.

IP version 6

IPv6 is version 6 of the Internet Protocol (IP), and is designed to replace the current IPv4 on the internet, but it is still not fully supported. Addresses use 128 bits, which are shown in groups of 4 hex digits:

xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx

IPv6 is classless. When subnetting, the notation /NN indicates the length of the network prefix. For instance:

2001:0db8:1234::/64

The notation '::' indicates one or more groups of value 0. There can only be one '::' in an address. The loopback is represented as ::1/128 or, simply, ::1.

IPv6 routing types are unicast, multicast, and anycast (the broadcast type does not exist in IPv6). Anycast allows you to send a message to a group where only one, the closest one, answers.

HTTP Tools

We have three tools for debugging HTTP protocols: netcat, curl, and browser inspector.

Netcat allows you to connect to a port and have a conversation, using pipes. If -u is specified use UDP, otherwise TCP. For example, to access the echo service on our machine:

$ nc localhost 7

CURL allows you to get the response from a URL on the network.

$ curl -I http://maripili.es
(GET, veure headers)
HTTP/1.1 200 OK
Date: Fri, 05 Apr 2019 05:03:23 GMT
Server: Apache
X-Logged-In: False
P3P: CP="NOI ADM DEV PSAi COM NAV OUR OTRo STP IND DEM"
Cache-Control: no-cache
Pragma: no-cache
Set-Cookie: 4af180c8954a0d5a1965b5b1b23ccbc5=pc1jg8f5kqigd71iec5a5lm7k5; path=/
X-Powered-By: PleskLin
Content-Type: text/html; charset=utf-8

$ curl http://maripili.es
(GET, content of the web page)

$ curl -v http://maripili.es
(GET, headers i contingut)

$ curl -d "key1=val1&key2=val2" http://maripili.es/contacto/
(POST)

Introduction to HTTP

HTTP is an application-level protocol for collaborative and distributed systems. It is the main component of the web, thanks to the use of hypertext documents. HTTP/1.1, the current version, is implemented using TCP in the transport. Version 2 is already standardized, and version 3 will work over UDP.

The secure version of HTTP is called HTTPS, or HTTP over TLS, the cryptographic protocol for secure transmission.

Session

A session is a sequence of requests/responses. It starts by establishing a TCP connection to a port on a server (usually 80). The server usually responds with a code, of the type "HTTP/1.1 200 OK", and with a body, which usually contains the requested resource.

HTTP is a stateless protocol, although some applications use mechanisms to store information. For example, cookies.

Messages

A request usually contains:

  • A request line, with a method. Example: GET /images/logo.png HTTP/1.1
  • Request header fields. Example: Accept-Language: ca
  • An empty line.
  • An optional body. Example: to make a POST.

Request Methods:

  • GET - The usual method for getting a resource. It has no body.
  • POST - The method used to send a body to the server. Used in forms.
  • PUT, DELETE, TRACE, OPTIONS, CONNECT, PATCH are other methods used.

A response contains:

  • A status line. Example: HTTP/1.1 200 OK.
  • Response header fields. Example: Content-Type: text/html
  • An empty line.
  • An optional body. Example: For a GET, the required content.

The status codes can be of the type:

  • Information (1XX).
  • Success (2XX). Example: 200 OK.
  • Redirection (3XX). Example: 301 Moved Permanently.
  • Client error (4XX). Example: 404 Not Found.
  • Server error (5XX). Example: 500 Internal Server Error.

We can use the telnet program to connect to an HTTP web server and send a GET command.

$ telnet maripili.es 80
Trying 217.160.0.165...
Connected to maripili.es.
Escape character is '^]'.
GET / HTTP/1.0
Host: maripili.es

This causes the server to respond:

HTTP/1.1 200 OK
Content-Type: text/html; charset=UTF-8
Connection: close
Date: Wed, 16 Feb 2022 08:12:31 GMT
Server: Apache

<!DOCTYPE html>
<html lang="es">
...
</html>
Connection closed by foreign host.

URL i HttpURLConnection

The URL class refers to a resource on the WWW. A generic resource can have the following form.

Let's see an example for the HTTP protocol:

https://www.example.com/test?key1=value1&key2=value2

In this case, we have to:

  • scheme is https
  • host is www.example.com
  • port is 80, but is not specified as it is the default value in the HTTP protocol
  • the path is test
  • the query is key1=value1&key2=value2

In Java, a URL can be constructed with:

URL url = new URL(String spec)

Once this is done, we can access each part of the URL using the getHost(), getPath(), getPort(), getProtocol(), getQuery(), etc. methods.

The two most important methods for interacting with the URL are:

  • URLConnection openConnection(): Returns a connection to the remote resource.
  • InputStream openStream(): Returns an InputStream to read the remote resource.

The class URLConnection is abstract, and if we have accessed an HTTP resource then the object will be an instance of HttpURLConnection.

openStream

To read a web page that is in the URL of a string called urlText, we can do:

URL url = new URL(urlText);
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));

To read a file:

BufferedInputStream in = new BufferedInputStream(new URL(urlText).openStream());

openConnection

With openConnection we can access the methods of the HTTP protocol and the status codes that are returned or the content type.

This is a GET method:

URL url = new URL(urlText);
HttpURLConnection httpConn = ((HttpURLConnection) url.openConnection());
httpConn.setRequestMethod("GET"); // optional: GET is the default method
int responseCode = httpConn.getResponseCode();
String contentType = httpConn.getContentType();
BufferedReader in = new BufferedReader(new InputStreamReader(httpConn.getInputStream()));
// in reading is left: a response from the server

This is a POST method:

URL url = new URL(urlText);
HttpURLConnection httpConn = ((HttpURLConnection) url.openConnection());
httpConn.setRequestMethod("POST");
httpConn.setDoOutput(true);
OutputStreamWriter out = new OutputStreamWriter(httpConn.getOutputStream());
out.write("propietat1=valor1&propietat2=valor2"); // values for the POST parameters
out.close();
int responseCode = httpConn.getResponseCode();
String contentType = httpConn.getContentType();
BufferedReader in = new BufferedReader(new InputStreamReader(httpConn.getInputStream()));
// in reading is left: a response from the server